Systems &amp; methods for intelligently managing multimedia for emergency response

ABSTRACT

Described herein are systems, devices, methods, and media for connecting a user for transmitting multimedia from electronic devices to emergency service providers. In some embodiments, a method for transmitting multimedia from an electronic device to an emergency service provider by an emergency management system includes the steps of: detecting an emergency alert indicative of an emergency; wherein each sensor within the set of pertinent sensors is determined to be associated with the emergency; obtaining a first set of multimedia contents from the set of pertinent sensors; determining, based on at least one of the set of pertinent sensors and the first set of multimedia contents, a set of relevant sensors from the set of pertinent sensors; and transmitting a second set of multimedia contents from the set of relevant sensors to an emergency service provider.

BACKGROUND OF THE INVENTION

A person in an emergency situation may request help using a mobilecommunication device such as a cell phone to dial a designated emergencynumber like 9-1-1 or a direct access phone number for the localemergency service provider (e.g., an emergency dispatch center). Thiscall is assigned to one or more first responders by the emergencyservice provider. However, these communications are typically limited toaudio calls with narrow functionality since most emergency serviceproviders that receive emergency calls currently lack the capacity formore sophisticated communications.

SUMMARY OF THE INVENTION

Disclosed herein are systems, servers, devices, methods, and media formanaging emergency communications such as multimedia datacommunications. The present disclosure takes advantage of recenttechnological advancements that have led to the dramatic rise in thenumber and popularity of sophisticated electronic devices in home,office, mobile, and outdoor environments—commonly referred to theInternet of Things (IoT)—that are capable of sensing and recordinginformation from the environment that they are placed in. In someembodiments, IoT devices, when in the vicinity of an emergency, areleveraged to provide valuable information, or insight, into theemergency. For example, smart cameras in the vicinity of an emergencymay be able to provide video streams of the environment surrounding theemergency. Emergency service providers may be able to use thisinformation to be better prepared to respond to emergencies and providelife-saving services to those involved.

One advantage provided by the systems, servers, devices, methods, andmedia of the instant application is the ability to identify sensors inthe vicinity of an emergency. In some embodiments, when a persongenerates an emergency alert (such as by dialing 9-1-1 on a mobile phonein the United States), an emergency management system (EMS) receives alocation of the mobile phone (e.g., the location of the emergency) fromthe mobile phone. In some embodiments, the EMS is configured to use thelocation, or any other data, received from the mobile phone to identifydevices that may be capable of recording information regarding theemergency. In some embodiments, the EMS then accesses the identifieddevices and receive data and multimedia collected by the identifieddevices. In some embodiments, the EMS facilitates the collection andtransfer of such data to emergency service providers (ESPs) such asemergency dispatch centers (EDCs) and/or public safety answering points(PSAPs).

Another advantage provided by the systems, servers, devices, methods,and media of the instant application is an intelligent multimedia systemcapable of intelligently determining the devices most relevant to anemergency. In some embodiments, after the EMS identifies devices thatmay be capable of recording information regarding an emergency, theintelligent multimedia system can determine, based on the devices andthe data collected by the devices, which of the devices identified bythe EMS may provide the most relevant information pertaining to theemergency. For example, in some embodiments, the intelligent multimediasystem determines which video camera from a set of three video camerasin the vicinity of an emergency has the best angle or clearest view ofthe emergency.

Another advantage provided by the systems, servers, devices, methods,and media of the instant application is the ability to transmitmultimedia received by the EMS to emergency service providers (ESPs),such as public safety answering points (PSAPs) and emergency responders(e.g., police officers or firemen). For example, in some embodiments,after receiving data or multimedia from a device in the vicinity of anemergency, the EMS identifies an ESP or a member of an ESP that isresponding to the emergency and transmit the data or multimedia to adevice associated with the EPS or member of the ESP. In someembodiments, the EMS then monitors and manages the transmission of thedata or multimedia to the ESP or member of the ESP. For example, in someembodiments, the EMS samples the connection strength of the transmissionand, in response to a weak connection strength, downgrade the quality ofthe transmission.

Another advantage provided by the systems, servers, devices, methods,and media of the instant application is a system for controllingemergency access to connected devices. For example, in some embodiments,an emergency management system (EMS) provides a web interface thatallows users to assign different levels or permissions of emergencyaccess to different electronic devices that the user possesses. Duringan emergency, the EMS accesses the electronic devices according to thelevels or permissions of emergency access assigned to the respectivedevices.

Another advantage provided by the systems, servers, devices, methods,and media of the instant application is the ability to requestmultimedia from a device or a user of the device and transmit multimediafrom the device to an emergency service provider (ESP). For example, insome embodiments, a member of an ESP can submit a multimedia requestregarding an emergency to an emergency management system (EMS), such asby selecting a multimedia access button within an emergency responseapplication. The EMS can then identify an electronic device associatedwith the emergency and deliver a multimedia inquiry to the electronicdevice. In response to receiving positive confirmation of the multimediainquiry, the EMS can then establish a communication link between theelectronic device and the ESP and transmit multimedia from theelectronic device to the ESP via the communication link.

In various embodiments, described herein are systems, servers, devices,methods, and media for transmitting multimedia from electronic devicesto emergency service providers. In one aspect, disclosed herein is amethod for transmitting multimedia from an electronic device to anemergency service provider (ESP) by an emergency management system(EMS), the method comprising: a) detecting an emergency alert indicativeof an emergency; b) identifying a set of pertinent sensors that isdetermined to be associated with the emergency based on at least onesensor attribute; c) obtaining a first set of multimedia contents fromthe set of pertinent sensors; d) determining a set of relevant sensorsbased on the first set of multimedia contents and optionally the atleast one sensor attribute; and e) transmitting a second set ofmultimedia contents from the set of relevant sensors to an emergencyservice provider (ESP). In some embodiments, the first set of multimediacontents comprises at least one of audio or video. In some embodiments,the first set of multimedia contents comprises at least one of stillimages or text files. In some embodiments, the first set of multimediacontents comprises at least one of audio or video feeds. In someembodiments, the ESP is a public safety answering point (PSAP). In someembodiments, the ESP is a dispatch center. In some embodiments,transmitting the second set of multimedia contents comprisestransmitting the second set of multimedia contents to a member of thedispatch center. In some embodiments, transmitting the second set ofmultimedia contents comprises delivering a weblink to a website hostingthe second set of multimedia contents to the ESP. In some embodiments,the ESP is a public safety answering point (PSAP) and the method furthercomprises: a) receiving, from a member of the PSAP, selection of a thirdset of multimedia contents from the second set of multimedia contents;and b) transmitting the third set of multimedia contents to anelectronic device associated with an emergency responder. In someembodiments, the ESP is a dispatch center and the method furthercomprises: a) receiving selection of a third set of multimedia contentsfrom the second set of multimedia contents by a member of the dispatchcenter; and b) transmitting the third set of multimedia contents to anelectronic device associated with an emergency responder. In someembodiments, the second set of multimedia contents is transmitted to theESP in response to a multimedia request received from a member of theESP. In some embodiments, the emergency alert comprises a locationassociated with the emergency; and the set of relevant sensors isdetermined based at least in part on distance between each sensor of theset of pertinent sensors and the location associated with the emergency.In some embodiments, determining the set of relevant sensors furthercomprises calculating a relevancy score for each multimedia content fromthe first set of multimedia contents. In some embodiments, calculating arelevancy score for each multimedia content from the first set ofmultimedia contents further comprises processing the first set ofmultimedia contents for at least one of audio or visual cues. In someembodiments, the audio cues comprise key words. In some embodiments, theaudio cues comprise volume level. In some embodiments, the visual cuescomprise at least one of light intensity, activity, objects, or people.In some embodiments, the method further comprises comparing an audio orvisual cue to a threshold value and discarding the audio or visual cueif the audio or visual cue fails to exceed the threshold value. In someembodiments, calculating a relevancy score further comprises: a)accessing a database of historical relevancy scores; b) obtaining amachine learning algorithm trained using data comprising the historicalrelevancy scores; and c) applying the machine learning algorithm to eachof the multimedia contents from the first set of multimedia contents tocalculate the relevancy score. In some embodiments, the method furthercomprises receiving relevancy feedback from the ESP regarding the secondset of multimedia contents and storing relevancy scores for the secondset of multimedia contents in the database of historical relevancyscores. In some embodiments, calculating the relevancy score furthercomprises obtaining and processing a transcription of a multimediacontent. In some embodiments, the set of relevant sensors is determinedby identifying each sensor from the set of pertinent sensors that has amultimedia content with a relevancy score that exceeds a thresholdrelevancy value. In some embodiments, the emergency alert comprises alocation associated with the emergency and the method further comprises:a) displaying a virtual map comprising indicators of at least one of thesecond set of multimedia contents or the set of relevant sensors shownin relation to the location associated with the emergency at anelectronic device associated with the ESP; and b) receiving, from amember of the ESP, selection of multimedia contents from at least one ofthe second set of multimedia contents or the set of relevant sensorsdisplayed on the virtual map. In some embodiments, the virtual mapfurther comprises an indicator of a relevancy score for the multimediacontent of each of the sensors from the set of relevant sensors. In someembodiments, the method further comprises receiving a radius ofrestriction on the set of relevant sensors through the virtual map fromthe member of the ESP and displaying only sensors located within theradius of restriction. In some embodiments, the method furthercomprises: a) receiving selection of a third set of multimedia contentsby the member of the ESP from the virtual map; and b) transmitting thethird set of multimedia contents to the ESP. In some embodiments, themethod further comprises: a) displaying a list of relevant sensors fromthe set of relevant sensors at an electronic device associated with theESP; and b) receiving selection of the second set of multimedia contentsby a member of the ESP from the list of relevant sensors. In someembodiments, the list of relevant sensors further comprises an indicatorof a relevancy score for multimedia content of each of the sensors fromthe set of relevant sensors. In some embodiments, the method furthercomprises: a) receiving selection of a third set of multimedia contentsby the member of the ESP from the list of relevant sensors; and b)transmitting the third set of multimedia contents from the set ofrelevant sensors to the ESP. In some embodiments, the method furthercomprises: a) sampling a connection quality of the transmission of amultimedia content within the second set of multimedia contents to theESP; and b) in response to the connection quality falling below athreshold value, downgrading the multimedia content. In someembodiments, the multimedia content is a video stream; and the videostream is downgraded to image stills or reduced frame rate orresolution. In some embodiments, the multimedia content is an audiostream; and the audio stream is downgraded to a text transcription. Insome embodiments, the text transcription is transmitted to the ESP inthe form of an SMS text message. In some embodiments, the multimediacontent is a video file; and the video file is downgraded to asummarization. In some embodiments, the method further comprises: a)processing the first set of multimedia contents for at least one ofaudio or visual cues; b) summarizing the at least one of the audio orvisual cues into one or more summarizations of the first set ofmultimedia contents; c) transmitting the one or more summarizations tothe ESP; and d) selecting the set of relevant sensors by receivingselection of the second set of multimedia contents by a member of theESP from the one or more summarizations. In some embodiments, the set ofpertinent sensors is a subset of sensors communicatively coupled to theEMS and continuously, periodically, or aperiodically providingmultimedia contents to the EMS. In some embodiments, the method furthercomprises establishing a communicative link with the set of pertinentsensors. In some embodiments, the method further comprises: a)determining, based on at least one of the set of pertinent sensors andthe first set of multimedia contents, a second set of relevant sensors;and b) transmitting a third set of multimedia contents from the secondset of relevant sensors to the ESP. In some embodiments, determining theset of relevant sensors further comprises calculating a relevancy scorefor each multimedia content within the first set of multimedia contents;and selecting the second set of relevant sensors further comprises: i)calculating a second relevancy score for each multimedia content fromthe first set of multimedia contents; and ii) selecting each sensor fromthe set of pertinent sensors that has a multimedia content with a secondrelevancy score that exceeds a threshold relevancy value as a relevantsensor. In some embodiments, the method further comprises: a)determining that one or more multimedia contents within the second setof multimedia contents is unavailable; and b) determining the second setof relevant sensors in response to determining that the one or more ofthe multimedia contents within the second set of multimedia contents isunavailable. In some embodiments, the emergency alert comprises alocation associated with the emergency; and identifying the set ofpertinent sensors comprises searching for sensors located within apredetermined radius of the location associated with the emergency. Insome embodiments, the emergency alert is received from an electronicdevice communicatively coupled to a network; and identifying the set ofpertinent sensors comprises detecting sensors communicatively coupled tothe network. In some embodiments, the emergency alert is received froman electronic device communicatively coupled to a network; andidentifying the set of pertinent sensors comprises detecting sensorsrecently communicatively coupled to the network. In some embodiments,the emergency alert is received from an electronic device; andidentifying the set of pertinent sensors comprises detecting sensorscommunicatively coupled to the electronic device. In some embodiments,identifying the set of pertinent sensors comprises parsing a sensordatabase. In some embodiments, the emergency alert is generated by asensor device. In some embodiments, the emergency alert is transmittedto the EMS by an electronic device communicatively coupled to the sensordevice. In some embodiments, the sensor device is a wearable device. Insome embodiments, the set of relevant sensors is determined at leastpartially by applying a prioritization rule to at least one of the setof pertinent sensors and the first set of multimedia contents. In someembodiments, the prioritization rule prioritizes the first set ofmultimedia contents based at least in part on content type. In someembodiments, the prioritization rule prioritizes the set of sensorspertinent to the emergency based at least in part on sensor type. Insome embodiments, the prioritization rule prioritizes the set of sensorspertinent to the emergency based at least in part on device type. Insome embodiments, the prioritization rule prioritizes the set of sensorspertinent to the emergency based at least in part on an emergency typeof the emergency. In some embodiments, the prioritization ruleprioritizes the first set of multimedia contents based at least in parton capabilities of the ESP user. In some embodiments, identifying theset of pertinent sensors comprises detecting active and inactive sensorsand selecting only active sensors as the set of pertinent sensors.

In another aspect, disclosed herein is a method for transmittingmultimedia from an electronic device to an emergency service provider(ESP) by an emergency management system (EMS), the method comprising: a)detecting an emergency alert indicative of an emergency from anelectronic device; b) identifying a set of pertinent sensors, whereineach sensor within the set of pertinent sensors is determined to beassociated with the emergency based on at least one sensor attribute; c)obtaining a first set of multimedia contents comprising multimediacontent from the set of pertinent sensors, wherein the multimediacontent comprises at least one of audio or video; d) determining, basedon at least one of the set of pertinent sensors and the first set ofmultimedia contents, a set of relevant sensors from the set of pertinentsensors; e) processing a second set of multimedia contents from the setof relevant sensors for audio or visual cues; f) summarizing the audioor visual cues into one or more summarizations of the second set ofmultimedia contents; and g) transmitting the one or more summarizationsto the ESP user.

In another aspect, disclosed herein is a method for transmittingmultimedia from an electronic device to an emergency service provider(ESP) by an emergency management system (EMS), the method comprising: a)detecting an emergency alert indicative of an emergency from anelectronic device, wherein the emergency alert comprises a locationassociated with the emergency; b) identifying a set of pertinentsensors, wherein each sensor within the set of pertinent sensors isdetermined to be associated with the emergency, by searching for sensorslocated within a predetermined radius of the location associated withthe emergency; c) obtaining a first set of multimedia contents from theset of pertinent sensors, wherein the first set of multimedia contentscomprises audio or video; d) determining, based on at least one of theset of sensors pertinent to the emergency and the first set ofmultimedia contents, a set of relevant sensors from the set of pertinentsensors; e) receiving a request for multimedia from an emergency serviceprovider (ESP); and f) in response to receiving the request formultimedia, transmitting a second set of multimedia contents from theset of relevant sensors to the ESP. In some embodiments, the methodfurther comprises: a) displaying a virtual map comprising at least oneof the second set of multimedia contents or the set of relevant sensorsshown in relation to the location associated with the emergency at anelectronic device associated with the ESP; and b) receiving, from amember of the ESP, selection of multimedia contents from at least one ofthe second set of multimedia contents or the set of relevant sensorsdisplayed on the virtual map. In some embodiments, the virtual mapfurther comprises an indicator of a relevancy score for the multimediacontent from each of the sensors from the set of relevant sensors. Insome embodiments, the method further comprises: a) sampling a connectionquality of the transmission of a multimedia content within the secondset of multimedia contents to the ESP; and b) in response to theconnection quality falling below a threshold value, downgrading themultimedia content. In some embodiments, the multimedia content is avideo stream; and the video stream is downgraded to image stills. Insome embodiments, the method further comprises: a) processing the firstset of multimedia contents for audio or visual cues; b) summarizing theaudio or visual cues into one or more summarizations of the first set ofmultimedia contents; c) transmitting the one or more summarizations tothe ESP; and d) selecting the set of relevant sensors by receivingselection of the second set of multimedia contents by a member of theESP from the one or more summarizations. In some embodiments,determining the set of relevant sensors further comprises calculating arelevancy score for each multimedia content within the first set ofmultimedia contents. In some embodiments, calculating a relevancy scorefor each multimedia content within the first set of multimedia contentsfurther comprises processing the first set of multimedia contents foraudio or visual cues. In some embodiments, the audio cues comprise keywords or volume level. In some embodiments, the visual cues compriselight intensity, activity, objects, or people. In some embodiments, themethod further comprises comparing an audio or visual cue to a thresholdvalue and discarding the audio or visual cue if the audio or visual cuefails to exceed the threshold value. In some embodiments, the methodfurther comprises: a) determining, based on at least one of the set ofpertinent sensors and the first set of multimedia contents, a second setof relevant sensors; and b) transmitting a third set of multimediacontents from the second set of relevant sensors to the ESP. In someembodiments, determining the set of relevant sensors further comprisescalculating a relevancy score for each multimedia content within thefirst set of multimedia contents; and selecting the second set ofrelevant sensors comprises: i) calculating a second relevancy score foreach multimedia content within the first set of multimedia contents; andii) selecting each sensor from the set of pertinent sensors having amultimedia content with a second relevancy score that exceeds athreshold relevancy value as a relevant sensor. In some embodiments, themethod further comprises: a) determining that one or more multimediacontents within the second set of multimedia contents is unavailable;and b) selecting the second set of relevant sensors in response todetermining that the one or more multimedia contents within the secondset of multimedia contents is unavailable. In some embodiments, theemergency alert is received from an electronic device communicativelycoupled to a network; and identifying the set of pertinent sensorscomprises detecting sensors communicatively coupled to the network. Insome embodiments, the emergency alert is received from an electronicdevice; and identifying the set of pertinent sensors comprises detectingsensors communicatively coupled to the electronic device.

In another aspect, disclosed herein is a system for transmittingmultimedia from an electronic device to an emergency service provider(ESP) by an emergency management system (EMS), the system comprising: a)a network server; b) an emergency service provider (ESP); and c) anemergency management system (EMS) communicatively coupled to the networkserver and the ESP, and configured to: i) detect an emergency alertindicative of an emergency; ii) identify a set of pertinent sensors,wherein each sensor within the set of pertinent sensors is determined tobe associated with the emergency based on at least one sensor attribute;iii) obtain a first set of multimedia contents from the set of pertinentsensors; iv) determine, based on the first set of multimedia contentsand optionally the at least one sensor attribute, a set of relevantsensors from the set of pertinent sensors; and v) transmit a second setof multimedia contents from the set of relevant sensors to the ESP. Insome embodiments, the first set of multimedia contents comprises atleast one of audio or video. In some embodiments, the first set ofmultimedia contents comprises at least one of still images or textfiles. In some embodiments, the first set of multimedia contentscomprises at least one of audio or video feeds. In some embodiments, theESP is a public safety answering point (PSAP). In some embodiments, theESP is a dispatch center. In some embodiments, the EMS is furtherconfigured to transmit the second set of multimedia contents to a memberof the dispatch center. In some embodiments, the EMS is furtherconfigured to deliver a weblink to a website hosting the second set ofmultimedia contents to the ESP. In some embodiments, the ESP is a publicsafety answering point (PSAP) and the EMS is further configured to: a)receive, from a member of the PSA, selection of a third set ofmultimedia contents from the second set of multimedia contents; and b)transmit the third set of multimedia contents to an electronic deviceassociated with an emergency responder. In some embodiments, the ESP isa dispatch center and the EMS is further configured to: a) receiveselection of a third set of multimedia contents from the second set ofmultimedia contents by a member of the dispatch center; and b) transmitthe third set of multimedia contents to an electronic device associatedwith an emergency responder. In some embodiments, the EMS transmits thesecond set of multimedia contents to the ESP in response to a multimediarequest received from a member of the ESP. In some embodiments, theemergency alert comprises a location associated with the emergency; andthe EMS determines the set of relevant sensors based at least in part ondistance between each sensor of the set of pertinent sensors and thelocation associated with the emergency. In some embodiments, the EMS isfurther configured to determine the set of relevant sensors bycalculating a relevancy score for each multimedia content from the firstset of multimedia contents. In some embodiments, the EMS is furtherconfigured to calculate a relevancy score for each multimedia contentfrom the first set of multimedia contents by processing the first set ofmultimedia contents for at least one of audio and visual cues. In someembodiments, the audio cues comprise key words. In some embodiments, theaudio cues comprise volume level. In some embodiments, the visual cuescomprise at least one of light intensity, activity, objects, or people.In some embodiments, the EMS is further configured to compare an audioor visual cue to a threshold value and discard the audio or visual cueif the audio or visual cue fails to exceed the threshold value. In someembodiments, the EMS is further configured to calculate a relevancyscore for each of the multimedia contents from the first set ofmultimedia contents by: a) accessing a database of historical relevancyscores; b) obtaining a machine learning algorithm trained using datacomprising the historical relevancy scores; and c) applying the machinelearning algorithm to each of the multimedia contents from the first setof multimedia contents. In some embodiments, the EMS is furtherconfigured to receive relevancy feedback from the ESP regarding thesecond set of multimedia contents and store relevancy scores for thesecond set of multimedia contents in the database of historicalrelevancy scores. In some embodiments, the EMS is further configured tocalculate the relevancy score by obtaining and processing atranscription of a multimedia content. In some embodiments, the EMS isfurther configured to determine the set of relevant sensors bydetermining each sensor from the set of pertinent sensors that has amultimedia content with a relevancy score that exceeds a thresholdrelevancy value. In some embodiments, the emergency alert comprises alocation associated with the emergency and the EMS is further configuredto: a) display a virtual map comprising indicators of at least one ofthe second set of multimedia contents or the set of relevant sensorsshown in relation to the location associated with the emergency at anelectronic device associated with the ESP; and b) receive, from a memberof the ESP, selection of multimedia contents from at least one of thesecond set of multimedia contents or the set of relevant sensorsdisplayed on the virtual map. In some embodiments, the virtual mapfurther comprises an indicator of a relevancy score for the multimediacontent of each of the sensors from the set of relevant sensors. In someembodiments, the EMS is further configured to receive a radius ofrestriction on the set of relevant sensors through the virtual map fromthe member of the ESP and displaying only sensors located within theradius of restriction. In some embodiments, the EMS is furtherconfigured to: a) receive selection of a third set of multimediacontents by the member of the ESP from the virtual map; and b) transmitthe third set of multimedia contents to the ESP. In some embodiments,the EMS is further configured to: a) display a list of relevant sensorsat an electronic device associated with the ESP; and b) receiveselection of the second set of multimedia contents by a member of theESP from the list of relevant sensors. In some embodiments, the list ofrelevant sensors further comprises an indicator of a relevancy score forthe multimedia content of each of the sensors from the set of relevantsensors. In some embodiments, the EMS is further configured to: a)receive selection of a third set of multimedia contents by the member ofthe ESP from the list of relevant sensors; and b) transmit the third setof multimedia contents from the set of relevant sensors to the ESP. Insome embodiments, the EMS is further configured to: a) sample aconnection quality of the transmission of a multimedia content withinthe second set of multimedia contents to the ESP; and b) in response tothe connection quality falling below a threshold value, downgrade themultimedia content. In some embodiments, the multimedia content is avideo stream; and the EMS downgrades the video stream to image stills orreduced frame rate or resolution. In some embodiments, the multimediacontent is an audio stream; and the EMS downgrades the audio stream to atext transcription. In some embodiments, the EMS transmits the texttranscription to the ESP in the form of an SMS text message. In someembodiments, the multimedia content is a video file; and the EMSdowngrades the video file to a summarization. In some embodiments, theEMS is further configured to: a) process the first set of multimediacontents for at least one of audio or visual cues; b) summarize the atleast one of the audio or visual cues into one or more summarizations ofthe first set of multimedia contents; c) transmit the one or moresummarizations to the ESP; and d) select the set of relevant sensors byreceiving selection of the second set of multimedia contents by a memberof the ESP from the one or more summarizations. In some embodiments, theset of pertinent sensors is a subset of sensors communicatively coupledto the EMS and continuously, periodically, or aperiodically providingmultimedia contents to the EMS. In some embodiments, the EMS is furtherconfigured to establish a communicative link with the set of pertinentsensors. In some embodiments, the EMS is further configured to: a)determine, based on at least one of the set of pertinent sensors and thefirst set of multimedia contents, a second set of relevant sensors; andb) transmit a third set of multimedia contents from the second set ofrelevant sensors to the ESP. In some embodiments, the EMS is furtherconfigured to determine the set of relevant sensors by calculating arelevancy score for each multimedia content within the first set ofmultimedia contents; and the EMS is further configured to select thesecond set of relevant sensors by: i) calculating a second relevancyscore for each of the multimedia content from the first set ofmultimedia contents; and ii) selecting each sensor from the set ofpertinent sensors that has a multimedia content with a second relevancyscore that exceeds a threshold relevancy value as a relevant sensor. Insome embodiments, the EMS is further configured to: a) determine thatone or more multimedia contents within the second set of multimediacontents is unavailable; and b) determine the second set of relevantsensors in response to determining that the one or more of themultimedia contents within the second set of multimedia contents isunavailable. In some embodiments, the emergency alert comprises alocation associated with the emergency; and the EMS is furtherconfigured to identify the set of pertinent sensors by searching forsensors located within a predetermined radius of the location associatedwith the emergency. In some embodiments, the EMS receives the emergencyalert from an electronic device communicatively coupled to a network;and the EMS identifies the set of pertinent sensors by detecting sensorscommunicatively coupled to the network. In some embodiments, the EMSreceives the emergency alert from an electronic device communicativelycoupled to a network; and the EMS identifies the set of pertinentsensors by detecting sensors communicatively recently coupled to thenetwork. In some embodiments, the EMS receives the emergency alert froman electronic device; and the EMS identifies the set of pertinentsensors by detecting sensors communicatively coupled to the electronicdevice. In some embodiments, the EMS is further configured to identifythe set of pertinent sensors by parsing a sensor database. In someembodiments, the emergency alert is generated by a sensor device. Insome embodiments, the EMS receives the emergency alert from electronicdevice communicatively coupled to the sensor device. In someembodiments, the sensor device is a wearable device. In someembodiments, the EMS is further configured to determine the set ofrelevant sensors at least partially by applying a prioritization rule toat least one of the set of pertinent sensors and the first set ofmultimedia contents. In some embodiments, the prioritization ruleprioritizes the first set of multimedia contents based at least in parton content type. In some embodiments, the prioritization ruleprioritizes the set of sensors pertinent to the emergency based at leastin part on sensor type. In some embodiments, the prioritization ruleprioritizes the set of sensors pertinent to the emergency based at leastin part on device type. In some embodiments, the prioritization ruleprioritizes the set of sensors pertinent to the emergency based at leastin part on an emergency type of the emergency. In some embodiments, theprioritization rule prioritizes the first set of multimedia contentsbased at least in part on capabilities of the ESP user. In someembodiments, the EMS is further configured to identify the set ofpertinent sensors by detecting active and inactive sensors and selectingonly active sensors as the set of pertinent sensors.

In another aspect, disclosed herein is a system for transmittingmultimedia from an electronic device to an emergency service provider(ESP) by an emergency management system (EMS), the system comprising: a)a network server; b) an emergency service provider (ESP); and c) anemergency management system (EMS) communicatively coupled to the networkserver and the ESP, and configured to: i) detect an emergency alertindicative of an emergency from an electronic device; ii) identify a setof pertinent sensors, wherein each sensor within the set of pertinentsensors is determined to be associated with the emergency based on atleast one sensor attribute; iii) obtain a first set of multimediacontents from the set of pertinent sensors; iv) determine, based on thefirst set of multimedia contents and optionally at least one sensorattribute, a set of relevant sensors from the set of pertinent sensors;v) process a second set of multimedia contents from the set of relevantsensors for audio or visual cues; vi) summarize the audio or visual cuesinto one or more summarizations of the second set of multimediacontents; and vii) transmit the one or more summarizations to the ESP.

In another aspect, disclosed herein is a system for transmittingmultimedia from an electronic device to an emergency service provider(ESP) by an emergency management system (EMS), the system comprising: a)a network server; b) an emergency service provider (ESP); and c) anemergency management system (EMS) communicatively coupled to the networkserver and the ESP, and configured to: i) detect an emergency alertindicative of an emergency from an electronic device, wherein theemergency alert comprises a location associated with the emergency; ii)identify a set of pertinent sensors, wherein each sensor within the setof pertinent sensors is determined to be associated with the emergency,by searching for sensors located within a predetermined radius of thelocation associated with the emergency; iii) obtain a first set ofmultimedia contents from the set of pertinent sensors, wherein the firstset of multimedia contents comprises audio or video; iv) determine,based on at least one of the set of pertinent sensors and the first setof multimedia contents, a set of relevant sensors from the set ofpertinent sensors; v) receive a request for multimedia from the ESP; andvi) in response to receiving the request for multimedia, transmit asecond set of multimedia contents from the set of relevant sensors tothe ESP. In some embodiments, the EMS is further configured to: a)display a virtual map comprising at least one of the second set ofmultimedia contents or the set of relevant sensors shown in relation tothe location associated with the emergency at an electronic deviceassociated with the ESP; and b) receive, from a member of the ESP,selection of multimedia contents from at least one of the second set ofmultimedia contents or the set of relevant sensors displayed on thevirtual map. In some embodiments, the virtual map further comprises anindicator of a relevancy score for multimedia content from each sensorfrom the set of relevant sensors. In some embodiments, the EMS isfurther configured to: a) sample a connection quality of thetransmission of a multimedia content within the second set of multimediacontents to the ESP; and b) in response to the connection qualityfalling below a threshold value, downgrade the multimedia content. Insome embodiments, the multimedia content is a video stream; and the EMSdowngrades the video stream to image stills. In some embodiments, theEMS is further configured to: a) process the first set of multimediacontents for audio or visual cues; b) summarize the audio or visual cuesinto one or more summarizations of the first set of multimedia contents;c) transmit the one or more summarizations to the ESP; and d) select theset of relevant sensors by receiving selection of the second set ofmultimedia contents by a member of the ESP from the one or moresummarizations. In some embodiments, the EMS is further configured todetermine the set of relevant sensors by calculating a relevancy scorefor each multimedia content within the first set of multimedia contents.In some embodiments, the EMS is further configured to calculate arelevancy score for each multimedia content within the first set ofmultimedia contents by processing the first set of multimedia contentsfor audio or visual cues. In some embodiments, the audio cues comprisekey words or volume level. In some embodiments, the visual cues compriselight intensity, activity, objects, or people. In some embodiments, theEMS is further configured to compare an audio or visual cue to athreshold value and discard the audio or visual cue if the audio orvisual cue fails to exceed the threshold value. In some embodiments, theEMS is further configured to: a) determine, based on at least one of theset of pertinent sensors and the first set of multimedia contents, asecond set of relevant sensors; and b) transmit a third set ofmultimedia contents from the second set of relevant sensors to the ESP.In some embodiments, the EMS is further configured to determine the setof relevant sensors by calculating a relevancy score for each multimediacontent within the first set of multimedia contents; and the EMS isfurther configured to determine the second set of relevant sensors by:i) calculating a second relevancy score for each multimedia contentwithin the first set of multimedia contents; and ii) selecting eachsensor from the set of pertinent sensors having a multimedia contentwith a second relevancy score that exceeds a threshold relevancy valueas a relevant sensor. In some embodiments, the EMS is further configuredto: a) determine that one or more multimedia contents within the secondset of multimedia contents is unavailable; and b) select the second setof relevant sensors in response to determining that the one or moremultimedia contents within the second set of multimedia contents isunavailable. In some embodiments, the EMS receives the emergency alertfrom an electronic device communicatively coupled to a network; and theEMS is further configured to identify the set of pertinent sensors bydetecting sensors communicatively coupled to the network. In someembodiments, the EMS receives the emergency alert from an electronicdevice; and the EMS is further configured to identify the set ofpertinent sensors by detecting sensors communicatively coupled to theelectronic device.

In another aspect, disclosed herein is a non-transitory computerreadable storage media encoded with a computer program includinginstructions executable by at least one processor for: a) detecting anemergency alert indicative of an emergency; b) identifying a set ofpertinent sensors, wherein each sensor within the set of pertinentsensors is determined to be associated with the emergency; c) obtaininga first set of multimedia contents from the set of pertinent sensors; d)determining, based on at least one of the set of pertinent sensors andthe first set of multimedia contents, a set of relevant sensors from theset of pertinent sensors; and e) transmitting a second set of multimediacontents from the set of relevant sensors to an emergency serviceprovider (ESP). In some embodiments, the first set of multimediacontents comprises at least one of audio or video. In some embodiments,the first set of multimedia contents comprises at least one of stillimages or text files. In some embodiments, the first set of multimediacontents comprises at least one of audio or video feeds. In someembodiments, the ESP is a public safety answering point (PSAP). In someembodiments, the ESP is a dispatch center. In some embodiments,transmitting the second set of multimedia contents comprisestransmitting the second set of multimedia contents to a member of thedispatch center. In some embodiments, transmitting the second set ofmultimedia contents comprises delivering a weblink to a website hostingthe second set of multimedia contents to the ESP. In some embodiments,the ESP is a public safety answering point (PSAP) and the media furtherincludes instructions for: a) receiving, from a member of the PSAP,selection of a third set of multimedia contents from the second set ofmultimedia contents; and b) transmitting the third set of multimediacontents to an electronic device associated with an emergency responder.In some embodiments, the ESP is a dispatch center and the media furtherincludes instructions for: a) receiving selection of a third set ofmultimedia contents from the second set of multimedia contents by amember of the dispatch center; and b) transmitting the third set ofmultimedia contents to an electronic device associated with an emergencyresponder. In some embodiments, the second set of multimedia contents istransmitted to the ESP in response to a multimedia request received froma member of the ESP. In some embodiments, the emergency alert comprisesa location associated with the emergency; and the set of relevantsensors is determined based at least in part on distance between eachsensor of the set of pertinent sensors and the location associated withthe emergency. In some embodiments, determining the set of relevantsensors further comprises calculating a relevancy score for eachmultimedia content from the first set of multimedia contents. In someembodiments, calculating a relevancy score for each multimedia contentfrom the first set of multimedia contents further comprises processingthe first set of multimedia contents for at least one of audio or visualcues. In some embodiments, the audio cues comprise key words. In someembodiments, the audio cues comprise volume level. In some embodiments,the visual cues comprise at least one of light intensity, activity,objects, or people. In some embodiments, the media further includesinstructions for comparing an audio or visual cue to a threshold valueand discarding the audio or visual cue if the audio or visual cue failsto exceed the threshold value. In some embodiments, calculating arelevancy score further comprises: a) accessing a database of historicalrelevancy scores; b) obtaining a machine learning algorithm trainedusing data comprising the historical relevancy scores; and c) applyingthe machine learning algorithm to each of the multimedia contents fromthe first set of multimedia contents to calculate the relevancy score.In some embodiments, the media further includes instructions forreceiving relevancy feedback from the ESP regarding the second set ofmultimedia contents and storing relevancy scores for the second set ofmultimedia contents in the database of historical relevancy scores. Insome embodiments, calculating the relevancy score further comprisesobtaining and processing a transcription of a multimedia content. Insome embodiments, the set of relevant sensors is determined bydetermining each sensor from the set of pertinent sensors that has amultimedia content with a relevancy score that exceeds a thresholdrelevancy value. In some embodiments, the emergency alert comprises alocation associated with the emergency and the media further comprisesinstructions for: a) displaying a virtual map comprising indicators ofat least one of the second set of multimedia contents or the set ofrelevant sensors shown in relation to the location associated with theemergency at an electronic device associated with the ESP; and b)receiving, from a member of the ESP, selection of multimedia contentsfrom at least one of the second set of multimedia contents or the set ofrelevant sensors displayed on the virtual map. In some embodiments, thevirtual map further comprises an indicator of a relevancy score for themultimedia content of each of the sensors from the set of relevantsensors. In some embodiments, the media further includes instructionsfor receiving a radius of restriction on the set of relevant sensorsthrough the virtual map from the member of the ESP and displaying onlysensors located within the radius of restriction. In some embodiments,the media further includes instructions for: a) receiving selection of athird set of multimedia contents by the member of the ESP from thevirtual map; and b) transmitting the third set of multimedia contents tothe ESP. In some embodiments, the media further includes instructionsfor: a) displaying a list of relevant sensors at an electronic deviceassociated with the ESP; and b) receiving selection of the second set ofmultimedia contents by a member of the ESP from the list of relevantsensors. In some embodiments, the list of relevant sensors furthercomprises an indicator of a relevancy score for multimedia content ofeach of the sensors from the set of relevant sensors. In someembodiments, the media further includes instructions for: a) receivingselection of a third set of multimedia contents by the member of the ESPfrom the list of relevant sensors; and b) transmitting the third set ofmultimedia contents from the set of relevant sensors to the ESP. In someembodiments, the media further includes instructions for: a) sampling aconnection quality of the transmission of a multimedia content withinthe second set of multimedia contents to the ESP; and b) in response tothe connection quality falling below a threshold value, downgrading themultimedia content. In some embodiments, the multimedia content is avideo stream; and the video stream is downgraded to image stills orreduced frame rate or resolution. In some embodiments, the multimediacontent is an audio stream; and the audio stream is downgraded to a texttranscription. In some embodiments, the text transcription istransmitted to the ESP in the form of an SMS text message. In someembodiments, the multimedia content is a video file; and the video fileis downgraded to a summarization. In some embodiments, the media furtherincludes instructions for: a) processing the first set of multimediacontents for at least one of audio or visual cues; b) summarizing the atleast one of the audio or visual cues into one or more summarizations ofthe first set of multimedia contents; c) transmitting the one or moresummarizations to the ESP; and d) selecting the set of relevant sensorsby receiving selection of the second set of multimedia contents by amember of the ESP from the one or more summarizations. In someembodiments, the set of pertinent sensors is a subset of sensorscommunicatively coupled to the EMS and continuously, periodically, oraperiodically providing multimedia contents to the EMS. In someembodiments, the media further includes instructions for establishing acommunicative link with the set of pertinent sensors. In someembodiments, the media further includes instructions for: a)determining, based on at least one of the set of pertinent sensors andthe first set of multimedia contents, a second set of relevant sensors;and b) transmitting a third set of multimedia contents from the secondset of relevant sensors to the ESP. In some embodiments, determining theset of relevant sensors further comprises calculating a relevancy scorefor each multimedia content within the first set of multimedia contents;and selecting the second set of relevant sensors further comprises: i)calculating a second relevancy score for each multimedia content fromthe first set of multimedia contents; and ii) selecting each sensor fromthe set of pertinent sensors that has a multimedia content with a secondrelevancy score that exceeds a threshold relevancy value as a relevantsensor. In some embodiments, the media further includes instructionsfor: a) determining that one or more multimedia contents within thesecond set of multimedia contents is unavailable; and b) determining thesecond set of relevant sensors in response to determining that the oneor more of the multimedia contents within the second set of multimediacontents is unavailable. In some embodiments, the emergency alertcomprises a location associated with the emergency; and identifying theset of pertinent sensors comprises searching for sensors located withina predetermined radius of the location associated with the emergency. Insome embodiments, the emergency alert is received from an electronicdevice communicatively coupled to a network; and identifying the set ofpertinent sensors comprises detecting sensors communicatively coupled tothe network. In some embodiments, the emergency alert is received froman electronic device communicatively coupled to a network; andidentifying the set of pertinent sensors comprises detecting sensorsrecently communicatively coupled to the network. In some embodiments,the emergency alert is received from an electronic device; andidentifying the set of pertinent sensors comprises detecting sensorscommunicatively coupled to the electronic device. In some embodiments,identifying the set of pertinent sensors comprises parsing a sensordatabase. In some embodiments, the emergency alert is generated by asensor device. In some embodiments, the emergency alert is transmittedto the EMS by an electronic device communicatively coupled to the sensordevice. In some embodiments, the sensor device is a wearable device. Insome embodiments, the set of relevant sensors is determined at leastpartially by applying a prioritization rule to at least one of the setof pertinent sensors and the first set of multimedia contents. In someembodiments, the prioritization rule prioritizes the first set ofmultimedia contents based at least in part on content type. In someembodiments, the prioritization rule prioritizes the set of sensorspertinent to the emergency based at least in part on sensor type. Insome embodiments, the prioritization rule prioritizes the set of sensorspertinent to the emergency based at least in part on device type. Insome embodiments, the prioritization rule prioritizes the set of sensorspertinent to the emergency based at least in part on an emergency typeof the emergency. In some embodiments, the prioritization ruleprioritizes the first set of multimedia contents based at least in parton capabilities of the ESP user. In some embodiments, identifying theset of pertinent sensors comprises detecting active and inactive sensorsand selecting only active sensors as the set of pertinent sensors.

In another aspect, disclosed herein is a non-transitory computerreadable storage media encoded with a computer program includinginstructions executable by at least one processor for: a) detecting anemergency alert indicative of an emergency from an electronic device; b)identifying a set of pertinent sensors, wherein each sensor within theset of pertinent sensors is determined to be associated with theemergency; c) obtaining a first set of multimedia contents comprisingmultimedia content from the set of pertinent sensors, wherein themultimedia content comprises at least one of audio or video; d)determining, based on at least one of the set of pertinent sensors andthe first set of multimedia contents, a set of relevant sensors from theset of pertinent sensors; e) processing a second set of multimediacontents from the set of relevant sensors for audio or visual cues; f)summarizing the audio or visual cues into one or more summarizations ofthe second set of multimedia contents; and g) transmitting the one ormore summarizations to the ESP user.

In another aspect, disclosed herein is a non-transitory computerreadable storage media encoded with a computer program includinginstructions executable by at least one processor for: a) detecting anemergency alert indicative of an emergency from an electronic device,wherein the emergency alert comprises a location associated with theemergency; b) identifying a set of pertinent sensors, wherein eachsensor within the set of pertinent sensors is determined to beassociated with the emergency, by searching for sensors located within apredetermined radius of the location associated with the emergency; c)obtaining a first set of multimedia contents from the set of pertinentsensors, wherein the first set of multimedia contents comprises audio orvideo; d) determining, based on at least one of the set of sensorspertinent to the emergency and the first set of multimedia contents, aset of relevant sensors from the set of pertinent sensors; e) receivinga request for multimedia from an emergency service provider (ESP); andf) in response to receiving the request for multimedia, transmitting asecond set of multimedia contents from the set of relevant sensors tothe ESP. In some embodiments, the media further includes instructionsfor: a) displaying a virtual map comprising at least one of the secondset of multimedia contents or the set of relevant sensors shown inrelation to the location associated with the emergency at an electronicdevice associated with the ESP; and b) receiving, from a member of theESP, selection of multimedia contents from at least one of the secondset of multimedia contents or the set of relevant sensors displayed onthe virtual map. In some embodiments, the virtual map further comprisesan indicator of a relevancy score for the multimedia content from eachof the sensors from the set of relevant sensors. In some embodiments,the media further includes instructions for: a) sampling a connectionquality of the transmission of a multimedia content within the secondset of multimedia contents to the ESP; and b) in response to theconnection quality falling below a threshold value, downgrading themultimedia content. In some embodiments, the multimedia content is avideo stream; and the video stream is downgraded to image stills. Insome embodiments, the media further includes instructions for: a)processing the first set of multimedia contents for audio or visualcues; b) summarizing the audio or visual cues into one or moresummarizations of the first set of multimedia contents; c) transmittingthe one or more summarizations to the ESP; and d) selecting the set ofrelevant sensors by receiving selection of the second set of multimediacontents by a member of the ESP from the one or more summarizations. Insome embodiments, determining the set of relevant sensors furthercomprises calculating a relevancy score for each multimedia contentwithin the first set of multimedia contents. In some embodiments,calculating a relevancy score for each multimedia content within thefirst set of multimedia contents further comprises processing the firstset of multimedia contents for audio or visual cues. In someembodiments, the audio cues comprise key words or volume level. In someembodiments, the visual cues comprise light intensity, activity,objects, or people. In some embodiments, the media further includesinstructions for comparing an audio or visual cue to a threshold valueand discarding the audio or visual cue if the audio or visual cue failsto exceed the threshold value. In some embodiments, the media furtherincludes instructions for: a) determining, based on at least one of theset of pertinent sensors and the first set of multimedia contents, asecond set of relevant sensors; and b) transmitting a third set ofmultimedia contents from the second set of relevant sensors to the ESP.In some embodiments, determining the set of relevant sensors furthercomprises calculating a relevancy score for each multimedia contentwithin the first set of multimedia contents; and selecting the secondset of relevant sensors comprises: i) calculating a second relevancyscore for each multimedia content within the first set of multimediacontents; and ii) selecting each sensor from the set of pertinentsensors having a multimedia content with a second relevancy score thatexceeds a threshold relevancy value as a relevant sensor. In someembodiments, the media further includes instructions for: a) determiningthat one or more multimedia contents within the second set of multimediacontents is unavailable; and b) selecting the second set of relevantsensors in response to determining that the one or more multimediacontents within the second set of multimedia contents is unavailable. Insome embodiments, the emergency alert is received from an electronicdevice communicatively coupled to a network; and identifying the set ofpertinent sensors comprises detecting sensors communicatively coupled tothe network. In some embodiments, the emergency alert is received froman electronic device; and identifying the set of pertinent sensorscomprises detecting sensors communicatively coupled to the electronicdevice.

In another aspect, disclosed herein is a computer-implemented system fortransmitting multimedia from an electronic device to an emergencyservice provider (ESP), the system comprising: a) a network servercomprising one or more processors; and b) an emergency management system(EMS) executed on the network server and configured to: i) identify anelectronic device as being associated with an emergency alert; ii)deliver a multimedia inquiry to the electronic device; iii) receive aconfirmation of the multimedia inquiry from the electronic device; iv)determine the emergency service provider (ESP) to receive multimediafrom the electronic device based on a location of the electronic deviceand a geofence system; v) establish a communication link between theelectronic device and the ESP; and vi) transmit multimedia content fromthe electronic device to the ESP via the communication link. In someembodiments, the multimedia content comprises a video feed. In someembodiments, the EMS is configured to transmit the multimedia contentfrom the electronic device to the ESP via the communication link by: a)hosting the video feed at a remote server; and b) providing the ESP witha web link to the video feed hosted on the remote server. In someembodiments, the EMS is configured to identify the electronic device asassociated with the emergency alert by autonomously receiving theemergency alert from the electronic device when the electronic deviceexecutes an emergency phone call. In some embodiments, the EMS isfurther configured to: a) display a multimedia access button within anemergency response application executed on a computing device at theESP; b) establish the communication link between the electronic deviceand the ESP in response to receiving selection of the multimedia accessbutton; and c) display the multimedia content through the emergencyresponse application. In some embodiments, the EMS is further configuredto: a) display a multimedia access button within an emergency responseapplication executed on a computing device at the ESP; b) receiveselection of the multimedia access button; and c) deliver the multimediainquiry to the electronic device in response to receiving the selectionof the multimedia access button. In some embodiments, the emergencyresponse application is a web application accessible via a web browserusing a URL. In some embodiments, the EMS is configured to identify theelectronic device as being associated with the emergency alert byreceiving an emergency data request from the ESP, wherein the emergencydata request is generated and transmitted by the emergency responseapplication in response to receiving selection of the multimedia accessbutton and wherein the emergency data request comprises an identifier ofthe electronic device. In some embodiments, the emergency alert is anemergency phone call made from the electronic device and wherein the EMSis configured to identify the electronic device as associated with theemergency alert by autonomously detecting the emergency phone call madeby the electronic device. In some embodiments, the EMS is configured todetermine the ESP to receive multimedia from the electronic device by:a) retrieving a plurality of geofences associated with a plurality ofESPs including the ESP; and b) determining that the location of theelectronic device is within a geofence associated with the ESP. In someembodiments, the EMS is further configured to: a) receive credentialsassociated with an account of an ESP user through an emergency responseapplication executed on a computing device at the ESP; b) identify anESP ID associated with the account of the ESP user; and c) determinethat the geofence associated with the ESP is associated with the ESP ID.In some embodiments, the multimedia inquiry is an SMS message comprisinga web link and wherein confirmation of the multimedia inquiry comprisesselection of the web link. In some embodiments, the multimedia inquiryis a push notification and wherein confirmation of the multimediainquiry comprises selection of the push notification. In someembodiments, the EMS is further configured to: a) sample a connectionquality of the transmission of the multimedia content; and b) inresponse to the connection quality falling below a threshold value,downgrading the multimedia content. In some embodiments: a) themultimedia content comprises a video feed; and b) the video feed isdowngraded to image stills or reduced framerate or resolution. In someembodiments: a) the multimedia content comprises a video file; and b)the video file is downgraded to a summarization. In some embodiments: a)the multimedia content comprises an audio feed or audio file; and b) theaudio feed or audio file is downgraded to a text transcription. In someembodiments, the EMS is further configured to determine an access levelassigned to the electronic device and transmit the multimedia inquiry tothe electronic device in response to determining that the EMS does nothave permission to access the electronic device based on the accesslevel assigned to the electronic device.

In another aspect, disclosed herein is a method for transmittingmultimedia from an electronic device to an emergency service provider(ESP) by an emergency management system (EMS), the method comprising: a)identifying an electronic device associated with an emergency alert; b)delivering a multimedia inquiry to the electronic device; c) receiving aconfirmation of the multimedia inquiry from the electronic device; d)determining an emergency service provider (ESP) to receive multimediafrom the electronic device using a location of the electronic device anda geofence system; e) establishing a communication link between theelectronic device and the ESP; and f) transmitting multimedia contentfrom the electronic device to the ESP via the communication link. Insome embodiments, the multimedia content comprises a video feed. In someembodiments, the EMS is configured to transmit the multimedia contentfrom the electronic device to the ESP via the communication link by: a)hosting the video feed at a remote server; and b) providing the ESP witha web link to the video feed hosted on the remote server. In someembodiments, the EMS is configured to identify the electronic device asassociated with the emergency alert by autonomously receiving theemergency alert from the electronic device when the electronic deviceexecutes an emergency phone call. In some embodiments, the EMS isfurther configured to: a) display a multimedia access button within anemergency response application executed on a computing device at theESP; b) establish the communication link between the electronic deviceand the ESP in response to receiving selection of the multimedia accessbutton; and c) display the multimedia content through the emergencyresponse application. In some embodiments, the EMS is further configuredto: a) display a multimedia access button within an emergency responseapplication executed on a computing device at the ESP; b) receiveselection of the multimedia access button; and c) deliver the multimediainquiry to the electronic device in response to receiving the selectionof the multimedia access button. In some embodiments, the emergencyresponse application is a web application accessible via a web browserusing a URL. In some embodiments, the EMS is configured to identify theelectronic device as being associated with the emergency alert byreceiving an emergency data request from the ESP, wherein the emergencydata request is generated and transmitted by the emergency responseapplication in response to receiving selection of the multimedia accessbutton and wherein the emergency data request comprises an identifier ofthe electronic device. In some embodiments, the emergency alert is anemergency phone call made from the electronic device and wherein the EMSis configured to identify the electronic device as associated with theemergency alert by autonomously detecting the emergency phone call madeby the electronic device. In some embodiments, the EMS is configured todetermine the ESP to receive multimedia from the electronic device by:a) retrieving a plurality of geofences associated with a plurality ofESPs including the ESP; and b) determining that the location of theelectronic device is within a geofence associated with the ESP. In someembodiments, the EMS is further configured to: a) receive credentialsassociated with an account of an ESP user through an emergency responseapplication executed on a computing device at the ESP; b) identify anESP ID associated with the account of the ESP user; and c) determinethat the geofence associated with the ESP is associated with the ESP ID.In some embodiments, the multimedia inquiry is an SMS message comprisinga web link and wherein confirmation of the multimedia inquiry comprisesselection of the web link. In some embodiments, the multimedia inquiryis a push notification and wherein confirmation of the multimediainquiry comprises selection of the push notification. In someembodiments, the EMS is further configured to: a) sample a connectionquality of the transmission of the multimedia content; and b) inresponse to the connection quality falling below a threshold value,downgrading the multimedia content. In some embodiments: a) themultimedia content comprises a video feed; and b) the video feed isdowngraded to image stills or reduced framerate or resolution. In someembodiments: a) the multimedia content comprises a video file; and b)the video file is downgraded to a summarization. In some embodiments: a)the multimedia content comprises an audio feed or audio file; and b) theaudio feed or audio file is downgraded to a text transcription. In someembodiments, the EMS is further configured to determine an access levelassigned to the electronic device and transmit the multimedia inquiry tothe electronic device in response to determining that the EMS does nothave permission to access the electronic device based on the accesslevel assigned to the electronic device.

In another aspect, disclosed herein is non-transitory computer readablemedium comprising instructions executable by a professor to create anapplication for transmitting multimedia from an electronic device to anemergency service provider (ESP) by an emergency management system(EMS), the application configured to: i) identify an electronic deviceas being associated with an emergency alert; ii) deliver a multimediainquiry to the electronic device; iii) receive a confirmation of themultimedia inquiry from the electronic device; iv) determine theemergency service provider (ESP) to receive multimedia from theelectronic device based on a location of the electronic device and ageofence system; v) establish a communication link between theelectronic device and the ESP; and vi) transmit multimedia content fromthe electronic device to the ESP via the communication link. In someembodiments, the multimedia content comprises a video feed. In someembodiments, the EMS is configured to transmit the multimedia contentfrom the electronic device to the ESP via the communication link by: a)hosting the video feed at a remote server; and b) providing the ESP witha web link to the video feed hosted on the remote server. In someembodiments, the EMS is configured to identify the electronic device asassociated with the emergency alert by autonomously receiving theemergency alert from the electronic device when the electronic deviceexecutes an emergency phone call. In some embodiments, the EMS isfurther configured to: a) display a multimedia access button within anemergency response application executed on a computing device at theESP; b) establish the communication link between the electronic deviceand the ESP in response to receiving selection of the multimedia accessbutton; and c) display the multimedia content through the emergencyresponse application. In some embodiments, the EMS is further configuredto: a) display a multimedia access button within an emergency responseapplication executed on a computing device at the ESP; b) receiveselection of the multimedia access button; and c) deliver the multimediainquiry to the electronic device in response to receiving the selectionof the multimedia access button. In some embodiments, the emergencyresponse application is a web application accessible via a web browserusing a URL. In some embodiments, the EMS is configured to identify theelectronic device as being associated with the emergency alert byreceiving an emergency data request from the ESP, wherein the emergencydata request is generated and transmitted by the emergency responseapplication in response to receiving selection of the multimedia accessbutton and wherein the emergency data request comprises an identifier ofthe electronic device. In some embodiments, the emergency alert is anemergency phone call made from the electronic device and wherein the EMSis configured to identify the electronic device as associated with theemergency alert by autonomously detecting the emergency phone call madeby the electronic device. In some embodiments, the EMS is configured todetermine the ESP to receive multimedia from the electronic device by:a) retrieving a plurality of geofences associated with a plurality ofESPs including the ESP; and b) determining that the location of theelectronic device is within a geofence associated with the ESP. In someembodiments, the EMS is further configured to: a) receive credentialsassociated with an account of an ESP user through an emergency responseapplication executed on a computing device at the ESP; b) identify anESP ID associated with the account of the ESP user; and c) determinethat the geofence associated with the ESP is associated with the ESP ID.In some embodiments, the multimedia inquiry is an SMS message comprisinga web link and wherein confirmation of the multimedia inquiry comprisesselection of the web link. In some embodiments, the multimedia inquiryis a push notification and wherein confirmation of the multimediainquiry comprises selection of the push notification. In someembodiments, the EMS is further configured to: a) sample a connectionquality of the transmission of the multimedia content; and b) inresponse to the connection quality falling below a threshold value,downgrading the multimedia content. In some embodiments: a) themultimedia content comprises a video feed; and b) the video feed isdowngraded to image stills or reduced framerate or resolution. In someembodiments: a) the multimedia content comprises a video file; and b)the video file is downgraded to a summarization. In some embodiments: a)the multimedia content comprises an audio feed or audio file; and b) theaudio feed or audio file is downgraded to a text transcription. In someembodiments, the EMS is further configured to determine an access levelassigned to the electronic device and transmit the multimedia inquiry tothe electronic device in response to determining that the EMS does nothave permission to access the electronic device based on the accesslevel assigned to the electronic device.

In another aspect, disclosed herein is a computer-implemented system fortransmitting multimedia from an electronic device to an emergencyservice provider (ESP), the system comprising: a) a network servercomprising one or more processors; and b) an emergency management system(EMS) executed on the network server and configured to: a) detect anemergency call made from an electronic device; b) in response todetecting the emergency call made from the electronic device, deliver amultimedia inquiry to the electronic device; c) receive a confirmationof the multimedia inquiry from the electronic device; d) obtain alocation of the electronic device; e) determine an emergency serviceprovider (ESP) to receive multimedia from the electronic device based onthe location of the electronic device; f) display a multimedia accessbutton within an emergency response application executed on a computingdevice at the ESP; and g) in response to receiving selection of themultimedia access button, establish a communication link between theelectronic device and the ESP and transmit multimedia content from theelectronic device to the ESP via the communication link. In someembodiments, the multimedia content comprises a video feed. In someembodiments, the EMS is configured to transmit the multimedia contentfrom the electronic device to the ESP via the communication link by: a)hosting the video feed at a remote server; and b) providing the ESP witha web link to the video feed hosted on the remote server. In someembodiments, the EMS is configured to identify the electronic device asassociated with the emergency alert by autonomously receiving theemergency alert from the electronic device when the electronic deviceexecutes an emergency phone call. In some embodiments, the EMS isfurther configured to: a) display a multimedia access button within anemergency response application executed on a computing device at theESP; b) establish the communication link between the electronic deviceand the ESP in response to receiving selection of the multimedia accessbutton; and c) display the multimedia content through the emergencyresponse application. In some embodiments, the EMS is further configuredto: a) display a multimedia access button within an emergency responseapplication executed on a computing device at the ESP; b) receiveselection of the multimedia access button; and c) deliver the multimediainquiry to the electronic device in response to receiving the selectionof the multimedia access button. In some embodiments, the emergencyresponse application is a web application accessible via a web browserusing a URL. In some embodiments, the EMS is configured to identify theelectronic device as being associated with the emergency alert byreceiving an emergency data request from the ESP, wherein the emergencydata request is generated and transmitted by the emergency responseapplication in response to receiving selection of the multimedia accessbutton and wherein the emergency data request comprises an identifier ofthe electronic device. In some embodiments, the emergency alert is anemergency phone call made from the electronic device and wherein the EMSis configured to identify the electronic device as associated with theemergency alert by autonomously detecting the emergency phone call madeby the electronic device. In some embodiments, the EMS is configured todetermine the ESP to receive multimedia from the electronic device by:a) retrieving a plurality of geofences associated with a plurality ofESPs including the ESP; and b) determining that the location of theelectronic device is within a geofence associated with the ESP. In someembodiments, the EMS is further configured to: a) receive credentialsassociated with an account of an ESP user through an emergency responseapplication executed on a computing device at the ESP; b) identify anESP ID associated with the account of the ESP user; and c) determinethat the geofence associated with the ESP is associated with the ESP ID.In some embodiments, the multimedia inquiry is an SMS message comprisinga web link and wherein confirmation of the multimedia inquiry comprisesselection of the web link. In some embodiments, the multimedia inquiryis a push notification and wherein confirmation of the multimediainquiry comprises selection of the push notification. In someembodiments, the EMS is further configured to: a) sample a connectionquality of the transmission of the multimedia content; and b) inresponse to the connection quality falling below a threshold value,downgrading the multimedia content. In some embodiments: a) themultimedia content comprises a video feed; and b) the video feed isdowngraded to image stills or reduced framerate or resolution. In someembodiments: a) the multimedia content comprises a video file; and b)the video file is downgraded to a summarization. In some embodiments: a)the multimedia content comprises an audio feed or audio file; and b) theaudio feed or audio file is downgraded to a text transcription. In someembodiments, the EMS is further configured to determine an access levelassigned to the electronic device and transmit the multimedia inquiry tothe electronic device in response to determining that the EMS does nothave permission to access the electronic device based on the accesslevel assigned to the electronic device.

In another aspect, disclosed herein is a method for transmittingmultimedia from an electronic device to an emergency service provider(ESP) by an emergency management system (EMS), the method comprising: a)detecting an emergency call made from an electronic device; b) inresponse to detecting the emergency call made from the electronicdevice, delivering a multimedia inquiry to the electronic device; c)receiving a confirmation of the multimedia inquiry from the electronicdevice; d) obtaining a location of the electronic device; e) determiningan emergency service provider (ESP) to receive multimedia from theelectronic device based on the location of the electronic device; f)displaying a multimedia access button within an emergency responseapplication executed on a computing device at the ESP; and g) inresponse to receiving selection of the multimedia access button,establishing a communication link between the electronic device and theESP and transmitting multimedia content from the electronic device tothe ESP via the communication link. In some embodiments, the multimediacontent comprises a video feed. In some embodiments, the EMS isconfigured to transmit the multimedia content from the electronic deviceto the ESP via the communication link by: a) hosting the video feed at aremote server; and b) providing the ESP with a web link to the videofeed hosted on the remote server. In some embodiments, the EMS isconfigured to identify the electronic device as associated with theemergency alert by autonomously receiving the emergency alert from theelectronic device when the electronic device executes an emergency phonecall. In some embodiments, the EMS is further configured to: a) displaya multimedia access button within an emergency response applicationexecuted on a computing device at the ESP; b) establish thecommunication link between the electronic device and the ESP in responseto receiving selection of the multimedia access button; and c) displaythe multimedia content through the emergency response application. Insome embodiments, the EMS is further configured to: a) display amultimedia access button within an emergency response applicationexecuted on a computing device at the ESP; b) receive selection of themultimedia access button; and c) deliver the multimedia inquiry to theelectronic device in response to receiving the selection of themultimedia access button. In some embodiments, the emergency responseapplication is a web application accessible via a web browser using aURL. In some embodiments, the EMS is configured to identify theelectronic device as being associated with the emergency alert byreceiving an emergency data request from the ESP, wherein the emergencydata request is generated and transmitted by the emergency responseapplication in response to receiving selection of the multimedia accessbutton and wherein the emergency data request comprises an identifier ofthe electronic device. In some embodiments, the emergency alert is anemergency phone call made from the electronic device and wherein the EMSis configured to identify the electronic device as associated with theemergency alert by autonomously detecting the emergency phone call madeby the electronic device. In some embodiments, the EMS is configured todetermine the ESP to receive multimedia from the electronic device by:a) retrieving a plurality of geofences associated with a plurality ofESPs including the ESP; and b) determining that the location of theelectronic device is within a geofence associated with the ESP. In someembodiments, the EMS is further configured to: a) receive credentialsassociated with an account of an ESP user through an emergency responseapplication executed on a computing device at the ESP; b) identify anESP ID associated with the account of the ESP user; and c) determinethat the geofence associated with the ESP is associated with the ESP ID.In some embodiments, the multimedia inquiry is an SMS message comprisinga web link and wherein confirmation of the multimedia inquiry comprisesselection of the web link. In some embodiments, the multimedia inquiryis a push notification and wherein confirmation of the multimediainquiry comprises selection of the push notification. In someembodiments, the EMS is further configured to: a) sample a connectionquality of the transmission of the multimedia content; and b) inresponse to the connection quality falling below a threshold value,downgrading the multimedia content. In some embodiments: a) themultimedia content comprises a video feed; and b) the video feed isdowngraded to image stills or reduced framerate or resolution. In someembodiments: a) the multimedia content comprises a video file; and b)the video file is downgraded to a summarization. In some embodiments: a)the multimedia content comprises an audio feed or audio file; and b) theaudio feed or audio file is downgraded to a text transcription. In someembodiments, the EMS is further configured to determine an access levelassigned to the electronic device and transmit the multimedia inquiry tothe electronic device in response to determining that the EMS does nothave permission to access the electronic device based on the accesslevel assigned to the electronic device.

In another aspect, disclosed herein is non-transitory computer readablemedium comprising instructions executable by a processor to create anapplication for transmitting multimedia from an electronic device to anemergency service provider (ESP) by an emergency management system(EMS), the application configured to: i) detect an emergency call madefrom an electronic device; ii) in response to detecting the emergencycall made from the electronic device, deliver a multimedia inquiry tothe electronic device; iii) receive a confirmation of the multimediainquiry from the electronic device; iv) obtain a location of theelectronic device; v) determine an emergency service provider (ESP) toreceive multimedia from the electronic device based on the location ofthe electronic device; vi) display a multimedia access button within anemergency response application executed on a computing device at theESP; and vii) in response to receiving selection of the multimediaaccess button, establish a communication link between the electronicdevice and the ESP and transmit multimedia content from the electronicdevice to the ESP via the communication link. In some embodiments, themultimedia content comprises a video feed. In some embodiments, the EMSis configured to transmit the multimedia content from the electronicdevice to the ESP via the communication link by: a) hosting the videofeed at a remote server; and b) providing the ESP with a web link to thevideo feed hosted on the remote server. In some embodiments, the EMS isconfigured to identify the electronic device as associated with theemergency alert by autonomously receiving the emergency alert from theelectronic device when the electronic device executes an emergency phonecall. In some embodiments, the EMS is further configured to: a) displaya multimedia access button within an emergency response applicationexecuted on a computing device at the ESP; b) establish thecommunication link between the electronic device and the ESP in responseto receiving selection of the multimedia access button; and c) displaythe multimedia content through the emergency response application. Insome embodiments, the EMS is further configured to: a) display amultimedia access button within an emergency response applicationexecuted on a computing device at the ESP; b) receive selection of themultimedia access button; and c) deliver the multimedia inquiry to theelectronic device in response to receiving the selection of themultimedia access button. In some embodiments, the emergency responseapplication is a web application accessible via a web browser using aURL. In some embodiments, the EMS is configured to identify theelectronic device as being associated with the emergency alert byreceiving an emergency data request from the ESP, wherein the emergencydata request is generated and transmitted by the emergency responseapplication in response to receiving selection of the multimedia accessbutton and wherein the emergency data request comprises an identifier ofthe electronic device. In some embodiments, the emergency alert is anemergency phone call made from the electronic device and wherein the EMSis configured to identify the electronic device as associated with theemergency alert by autonomously detecting the emergency phone call madeby the electronic device. In some embodiments, the EMS is configured todetermine the ESP to receive multimedia from the electronic device by:a) retrieving a plurality of geofences associated with a plurality ofESPs including the ESP; and b) determining that the location of theelectronic device is within a geofence associated with the ESP. In someembodiments, the EMS is further configured to: a) receive credentialsassociated with an account of an ESP user through an emergency responseapplication executed on a computing device at the ESP; b) identify anESP ID associated with the account of the ESP user; and c) determinethat the geofence associated with the ESP is associated with the ESP ID.In some embodiments, the multimedia inquiry is an SMS message comprisinga web link and wherein confirmation of the multimedia inquiry comprisesselection of the web link. In some embodiments, the multimedia inquiryis a push notification and wherein confirmation of the multimediainquiry comprises selection of the push notification. In someembodiments, the EMS is further configured to: a) sample a connectionquality of the transmission of the multimedia content; and b) inresponse to the connection quality falling below a threshold value,downgrading the multimedia content. In some embodiments: a) themultimedia content comprises a video feed; and b) the video feed isdowngraded to image stills or reduced framerate or resolution. In someembodiments: a) the multimedia content comprises a video file; and b)the video file is downgraded to a summarization. In some embodiments: a)the multimedia content comprises an audio feed or audio file; and b) theaudio feed or audio file is downgraded to a text transcription. In someembodiments, the EMS is further configured to determine an access levelassigned to the electronic device and transmit the multimedia inquiry tothe electronic device in response to determining that the EMS does nothave permission to access the electronic device based on the accesslevel assigned to the electronic device.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present invention will be obtained by reference to thefollowing detailed description that sets forth illustrative embodiments,in which the principles of the invention are utilized, and theaccompanying drawings of which:

FIG. 1A depicts diagrams of (i) an electronic device, (ii) an emergencymanagement system (EMS) in accordance with one embodiment of the presentdisclosure;

FIG. 1B depicts diagrams of (iii) an emergency service provider (ESP)system and (iv) ESP software in accordance with one embodiment of thepresent disclosure;

FIG. 2 depicts a diagram of a clearinghouse for emergency data inaccordance within one embodiment of the present disclosure;

FIG. 3 depicts a diagram of a geofence applied to a clearinghouse foremergency data in accordance with one embodiment of the presentdisclosure;

FIG. 4 depicts a diagram of an intelligent multimedia system inaccordance with one embodiment of the present disclosure;

FIG. 5 illustrates a map of devices in accordance with one embodiment ofthe present disclosure;

FIG. 6 depicts audio transcriptions in accordance with one embodiment ofthe present disclosure;

FIG. 7A, FIG. 7B, and FIG. 7C illustrate multimedia analysis processesin accordance with one embodiment of the present disclosure;

FIG. 8 depicts multimedia summarizations in accordance with oneembodiment of the present disclosure;

FIG. 9A and FIG. 9B depict ESP systems in accordance with one embodimentof the present disclosure;

FIG. 10A and FIG. 10B depict method flow diagrams in accordance with oneembodiment of the present disclosure; and

FIG. 11 depicts a method flow diagram in accordance with one embodimentof the present disclosure;

FIGS. 12A, 12B, 12C, and 12D illustrate examples of an emergency accessmanagement portal in accordance with one embodiment of the presentdisclosure;

FIG. 13 illustrates an example of a multimedia inquiry in accordancewith one embodiment of the present disclosure;

FIG. 14 illustrates an example of a mobile application for sharingemergency data and multimedia in accordance with one embodiment of thepresent disclosure;

FIG. 15 illustrates an example of multimedia inquiry in accordance withone embodiment of the present disclosure; and

FIGS. 16A and 16B illustrate an example of an emergency responseapplication in accordance with one embodiment of the present disclosure.

DETAILED DESCRIPTION

In certain embodiments, disclosed herein are devices, systems, andmethods for managing emergency data for emergency response. FIG. 1Adepicts diagrams of (i) an electronic device 110 and (ii) an emergencymanagement system (EMS) 120 in accordance with one embodiment of thepresent disclosure. In some embodiments, the electronic device 110 is adigital processing device such as a communication device (e.g., mobileor cellular phone, computer, laptop, etc.). In some embodiments, theelectronic device is a wearable device (e.g., a smartwatch). In someembodiments, the electronic device is an Internet of Things (IoT)device, such as a home assistant (e.g., an Amazon Echo) or a connectedsmoke detector (e.g., a Nest Protect smoke and carbon monoxide alarm).In some embodiments, the electronic device is a walkie-talkie or two-wayradio.

In some embodiments, the electronic device 110 includes a display 111, aprocessor 112, a memory 113 (e.g., an EPROM memory, a RAM, or asolid-state memory), and several optional components such as one or morenetwork component(s) 114 (e.g., an antenna and associated components,Wi-Fi adapters, Bluetooth adapters, etc.), a data storage 115, a userinterface 116, a computer program such as an emergency alert program117, one or more location components 118, and one or more sensors 119.In some embodiments, the processor 112 is implemented as one or moremicroprocessors, microcomputers, microcontrollers, digital signalprocessors, central processing units, state machines, logic circuitries,and/or devices that manipulate signals based on operationalinstructions. Among other capabilities, the processor 112 is configuredto fetch and execute computer-readable instructions stored in the memory113.

In some embodiments, the display 111 is part of the user interface 116(e.g., a touchscreen is both a display and a user interface in that itprovides an interface to receive user input or user interactions). Insome embodiments, the user interface 116 includes physical buttons suchas an on/off button or volume buttons. In some embodiments, the display111 and/or the user interface 116 comprises a touchscreen (e.g., acapacitive touchscreen), which is capable of displaying information andreceiving user input. In some embodiments, the user interface 116 doesnot include a touchscreen, but comprises one or more physical buttonsand/or a microphone. In some embodiments, the display 111 does notinclude a touchscreen, but comprises one or more lights, indicators,and/or lighted buttons.

In some embodiments, the electronic device 110 includes variousaccessories that allow for additional functionality. In someembodiments, these accessories (not shown) include one or more of thefollowing: a microphone, a camera, speaker, a fingerprintscanner/reader, health or environmental sensors, a USB or micro-USBport, a headphone jack, a card reader, a SIM card slot, or anycombination thereof. In some embodiments, the one or more sensorsinclude, but are not limited to: a gyroscope, an accelerometer, athermometer, a heart rate sensor, a barometer, or a hematology analyzer.In some embodiments, the data storage 115 includes a location data cache115 a and a user data cache 115 b. In some embodiments, the locationdata cache 115 a is configured to store locations generated by the oneor more location components 118.

In some embodiments, the computer program 117 is an emergency responseapplication or emergency response mobile application. In someembodiments, the computer program 117 is configured to record user data,such as a name, address, or medical data of a user associated with theelectronic device 110. In some embodiments, the computer program 117 isan emergency alert program configured to detect an emergency from theelectronic device 110 (e.g., when a user 100 (not shown) uses theelectronic device 110 to make an emergency call). In some embodiments,the user 100 initiates the emergency alert by interacting with the userinterface 116. In some embodiments, the emergency is detected by one ormore sensors (built in or peripheral to the device 110). In someembodiments, in response to detecting an emergency request forassistance (e.g., a native dial 9-1-1 call) generated or sent by theelectronic device 110, the computer program is configured to deliver anotification to the EMS 120. In some embodiments, the notification is anHTTP post containing information regarding the emergency request. Insome embodiments, the notification includes a location (e.g., adevice-based hybrid location) generated by or for the electronic device110. In some embodiments, in response to detecting an emergency requestgenerated or sent by the electronic device 110, the computer program isconfigured to deliver user data to the emergency management system (EMS)120.

In some embodiments, as depicted in FIG. 1A, the emergency managementsystem (EMS) 120 includes an EMS operating system 124, an EMS CPU 126,an EMS memory unit 127, and an EMS communication element 128. In someembodiments, the EMS CPU 126 is implemented as one or moremicroprocessors, microcomputers, microcontrollers, digital signalprocessors, central processing units, state machines, logic circuitries,and/or devices that manipulate signals based on operationalinstructions. Among other capabilities, the EMS CPU 126 is configured tofetch and execute computer-readable instructions stored in the EMSmemory unit 127. The EMS memory unit 127 optionally includes anycomputer-readable medium known in the art including, for example,volatile memory, such as static random-access memory (SRAM) and dynamicrandom-access memory (DRAM), and/or non-volatile memory, such asread-only memory (ROM), erasable programmable ROM, flash memories, harddisks, optical disks, and magnetic tapes. The EMS memory unit 127optionally includes modules, routines, programs, objects, components,data structures, etc., which perform particular tasks or implementparticular abstract data types.

In some embodiments, the EMS 120 includes one or more EMS databases 122,one or more servers 123, and an intelligent multimedia system 150. Insome embodiments, the EMS 120 includes an emergency response application180. In some embodiments, the intelligent multimedia system 150, asdescribed in further detail below, comprises an input/output (I/O)interface configured to manage communications and data transfers to andfrom the EMS 120 and external systems and devices. In some embodiments,the intelligent multimedia system 150 includes a variety of software andhardware interfaces, for example, a web interface, a graphical userinterface (GUI), and the like. In some embodiments, the EMS 120 includesone or more software modules 129. The intelligent multimedia system 150optionally enables the EMS 120 to communicate with other computingdevices, such as web servers and external data servers (not shown). Insome embodiments, the intelligent multimedia system 150 facilitatesmultiple communications within a wide variety of networks and protocoltypes, including wired networks, for example, LAN, cable, etc., andwireless networks, such as WLAN, cellular, or satellite. In someembodiments, the intelligent multimedia system 150 includes one or moreports for connecting a number of devices to one another or to anotherserver.

In some embodiments, as depicted in FIG. 1B, an ESP is an emergencyservice provider (ESP) system 130 includes one or more of a display 131,a processor 132, a memory 138, a user interface 136, at least onecentral processing unit or processor 132, a network component 135, anaudio system 134 (e.g., microphone, speaker and/or a call-takingheadset), and a computer program such as a ESP application or ESPprogram 139. In some embodiments, the ESP application or program 139comprises one or more software modules 140. In some embodiments, the ESPsystem 130 comprises a database of emergency responders 137, such asmedical assets, police assets, fire response assets, rescue assets,safety assets, etc.

In some embodiments, as depicted in FIG. 1B, the ESP application orprogram 139 installed on an ESP system 130 comprises at least onesoftware module 140 such as a call taking module 145 (e.g., a computeraided dispatch system), an ESP display module 146, a supplemental orupdated information module 147, a feedback module 148 (for ESP users torequest specific type of data), or a combination thereof. In someembodiments, the ESP application 139 displays the information on a map(e.g., on the display 131). In some embodiments, the location feedbackmodule 148 is configured to provide feedback from the ESP 130 to theEMS, as described below. In some embodiments, location and supplementalinformation is displayed for other emergency response providers (e.g.,police, fire, medical, etc.) and/or responders on their devices. It iscontemplated that responder devices have optionally installed aresponder device program (not shown) similar to ESP display module 146.In some embodiments, the responder device program displays the emergencylocation on a map.

Emergency Clearinghouse

In some embodiments, as depicted by FIG. 1A, the emergency managementsystem (EMS) 120 includes a clearinghouse 170 (also referred to as an“Emergency Clearinghouse”) for storing and retrieving emergency data. Insome embodiments, the clearinghouse 170 includes a locationclearinghouse 170A and an additional data clearinghouse 170B. In someembodiments, the location clearinghouse 170A includes a locationingestion module and a location retrieval module, as described belowwith respect to FIG. 2. In some embodiments, the additional dataclearinghouse 170B includes an additional data ingestion module and anadditional data retrieval module, as described below with respect toFIG. 2. In other embodiments, additional data and location data(hereinafter “emergency data”) are stored in one or more databases in adistributed manner. In some embodiments, the emergency data is stored inan external or third-party server that is accessible to the EMS 120. Theclearinghouse 170 optionally functions as an interface that receives andstores emergency data from electronic or communication devices that arethen retrieved, transmitted, and/or distributed to recipients (e.g.,emergency personnel) before, during, or after emergencies. As describedabove, the clearinghouse optionally receives emergency data fromelectronic or communication devices such as mobile phones, wearabledevices, laptop or desktop computers, personal assistants, intelligentvehicle systems, home security systems, IoT devices, camera feeds, andother sources. As described above and below, emergency data optionallyincludes locations or additional data such as medical history, personalinformation, or contact information. In some embodiments, during anemergency, the clearinghouse 170 detects the emergency and/or otherwiseidentifies the need to provide emergency data pertaining to theemergency. The clearinghouse 170 then identifies any emergency datapertaining to the emergency stored within the clearinghouse 170 andtransmits the pertinent emergency data to the requesting ESP.Accordingly, in some embodiments, the clearinghouse 170 acts as a datapipeline that automatically pushes emergency data to the ESP that wouldotherwise be without access to emergency data that is critical to mosteffectively and efficiently responding to an emergency. Accordingly,location data stored within the clearinghouse 170 allows emergencyresponders to arrive at the scene of an emergency faster, and additionaldata stored within the clearinghouse 170 allows emergency responders tobe better prepared for the emergencies they face.

For example, in one embodiment, an emergency alert is triggered by anelectronic device 110 (e.g., by pressing a soft button, a physicalbutton, voice command, or gesture) or autonomously based on sensor data(e.g., smoke alarms). In this example, the user then confirms theemergency and/or provides authorization for sending the emergency alert.Emergency data, such as an enhanced location and additional dataregarding the user (e.g., the user's medical history) is delivered bythe electronic device 110 to the EMS 120 and stored in the clearinghouse170 (e.g., in the location clearinghouse 170A and the additional dataclearinghouse 170B). In some embodiments, the EMS 120 or clearinghouse170 formats the emergency data into a format that is compatible withindustry standards for storing and sharing emergency data. For example,the emergency data is formatted to be compatible with National EmergencyNumber Association (NENA) standards. In some embodiments, theclearinghouse 170 transmits the emergency data to a receiving party inresponse to receiving a query from the receiving party, as describedbelow. In some embodiments, the clearinghouse 170 automatically pushesthe emergency data to a receiving party such as the PSAP. For example,in some embodiments, the emergency management system automaticallypushes the emergency data to a receiving party using a subscriptionsystem, as described below.

In some embodiments, as mentioned above, a requesting party (such as aPSAP responding to an emergency alert) queries the clearinghouse 170with an emergency data request (such as a HTTP GET request). In someembodiments, the emergency data request is in the form of the LocationInformation Server (LIS) protocol. In response to the emergency datarequest, the EMS 120 or clearinghouse 170 sends an appropriate responseincluding relevant emergency data to the requesting party via anencrypted pathway. In some embodiments, the emergency data request is inthe form of HTTP-Enabled Location Delivery (HELD) and the response fromthe EMS 120 or clearinghouse 170 is in the form of Presence InformationData Format Location Object (PIDF-LO). In some embodiments, theemergency data request includes an authorization code (also referred toas an “authorization token” or “temporary access token”) in the body,header, or metadata of the request, and the EMS 120 checks that theauthorization code is active before providing a response to therequesting party. In some embodiments, authorization is provided in the“Authorization” header of the emergency data request using HTTP BasicAuthentication. For example, in some embodiments, authorization isbase64-encoded user name and password for an account associated with therequesting party. In some embodiments, emergency data requests are sentover public networks using API access keys or credentials. In someembodiments, Transport Layer Security (TLS) is used in the requests andresponses from the EMS 120 for encryption security. In some embodiments,the call taking module 145 includes a call-handling application, whichis provided by a third-party vendor. In some embodiments, an ESPpersonnel interacts with the call-handling application to send anemergency data request to the EMS 120. In some embodiments, the responsefrom the EMS 120 is displayed at the ESP display 131.

In some embodiments, as described above, emergency data includeslocations and additional data. In some embodiments, emergency dataincludes one or more emergency data categories (also referred to as“data categories”). In some embodiments, the emergency data categoriesinclude: service data reference, full name, email, emergency contacts,addresses, language, occupation, phone numbers, websites, gender,height, weight, ethnicity, profile picture, allergies, medicalconditions, medications, disabilities, blood type, medical notes,birthday, and additional comments. In some embodiments, emergency datacategories are tagged with tags for specific types of data such as“demographics” or “medical data.” For example, in some embodiments,gender, height, weight, ethnicity, profile picture (image-url) aretagged as demographic data. In some embodiments, medical data protectedunder HIPAA and other laws are tagged as “HIPAA” or “private.” In someembodiments, medical data includes information on one or more ofallergies, medical condition(s) or illness(es), medication(s),disabilities, blood type, medical note(s), and other medicalinformation. In some embodiments, medical information protected underHIPAA are encrypted and/or anonymized. In some embodiments, some dataare tagged as “general” or another similar tag, wherein access is notspecifically restricted.

An example of an additional data communication from the EMS 120 in astandard format compatible with industry standards, PIDF-LO, is shownbelow.

HTTP/1.1 200 OK

Date: Tue, 1 Dec. 2016 23:27:30 GMT

Content-Length: 489

Content-Type: application/EmergencyCallData.DeviceInfo+xml

<dev:EmergencyCallData.DeviceInfo

xmlns:dev=“urnietf:params:xml:ns:EmergencyCallData:DeviceInfo”>

<dev:DataProviderReference>d4b3072df.201409182208075@example.org

In some embodiments, when the emergency data is stored at a third-partyserver and receives a request for emergency data from the EMS 120, as adatabase query, the third-party server formats the requested emergencydata and stores this information in an alternate database, and forwardseither a response or a reference to the alternate database for accessingthe emergency data requested by the EMS 120, which is provided to theESP 130 over a hybrid analog and/or a data communication channel,depending on the capabilities of ESP 130. In some embodiments, thethird-party server stores the emergency data, requested by the EMS 120or directly by the ESP 130, in the alternate database for a certainperiod of time after receiving the request for the emergency dataregarding a user and any electronic devices 110. In some embodiments,this period of time is a timer value (e.g., a timer countdown or a settime point) defined by the EMS 120 and the third-party server inconjunction with each other prior to the addition of the requestedemergency data to the alternate database at the third-party server. Insome embodiments, once the timer value has passed and no new requestsfor the emergency data pertaining to the particular user and theelectronic device 110, or other devices associated with the user, arereceived by the third-party server, then the third-party server marksthe particular alternate database entries to be deleted and waits foranother, different, time-out interval. In some embodiments, once thisparticular second time-out interval has also been completed and no newrequests for location data for the particular user or associatedelectronic devices 110 are received by the third-party server, thethird-party server removes the specific marked entries from thealternate database in the next cycle of updates for the alternatedatabase. In some embodiments, after adding the emergency data in thealternate database by the third-party server, the third-party serverkeeps updating the emergency data in the alternate database on aperiodic, or as-needed basis, for the purpose of keeping the emergencydata about the user or electronic device 110 current for providing themost recent and accurate emergency data to the EMS 120 and the ESP 130for the purposes of responding to a request for emergency assistance. Insome embodiments, the third-party server is updated by the EMS 120 forall the emergency data pertaining to all users and their associatedelectronic devices 110 that are served by the EMS 120 at any currenttime.

In some non-emergency situations, there is a need to access locationdata, user data, emergency data or sensor data. For example, in someembodiments, a user of an electronic device 110 grants authorization tofamily members to access location data for the user. Accordingly, when afamily member requests location data for a user, access is granted ifthere is proper authorization. As another example, in some embodiments,a taxi operations company requests and obtains location data of one ormore fleet members to keep track of its vehicles (e.g., via onboardvehicle console or terminal).

Various embodiments and applications of the clearinghouse 170 aredescribed in detail herein. However, the embodiments and applicationsdescribed herein should not be considered exhaustive or limiting in anyway.

FIG. 2 depicts an embodiment of an Emergency Clearinghouse 270 forstoring and retrieving emergency data. In some embodiments, theclearinghouse 270 includes a set of ingestion modules 278 (also referredto as “ingestion modules”) and a set of retrieval modules 279 (alsoreferred to as “retrieval modules”). The set of ingestion modules 278 isconfigured to receive various forms of emergency data from variousemergency data sources 262, such as an electronic device 210 or athird-party server system 261 (hereinafter, “third-party server”). Insome embodiments, an electronic device 210 is a communication device(e.g., a mobile phone), a wearable device (e.g., a smartwatch), or aninternet of things (IoT) device (e.g., a smart speaker) that cancommunicate with one or more of the ingestion modules within the set ofingestion modules 278. In some embodiments, a third-party server 261stores data that is not generated by or stored within an electronicdevice. For example, in some embodiments, a third-party server includesa database of static medical information that can be sent to theclearinghouse during an emergency. In some embodiments, when theemergency management system 120 detects an emergency (e.g., when aperson calls 9-1-1), the clearinghouse can query an emergency datasource 262 for emergency data regarding the emergency. For example, insome embodiments, in response to detecting a 9-1-1 call made from amobile phone, the additional data ingestion module 272 (as describedbelow) sends a query including the phone number of the mobile phone to athird-party server 261 that stores static medical information. Thethird-party server 261 can then return any available medical informationassociated with the phone number of the mobile phone to the additionaldata ingestion module. In some embodiments, multiple ingestion moduleswithin the set of ingestion modules can receive emergency data for asingle emergency. For example, in some embodiments, when a person calls9-1-1 from a mobile phone, the mobile phone can send a device-basedhybrid location to the location ingestion module 271 (as describedbelow) and demographic data (as described above) to the additional dataingestion module 272. In some embodiments, the clearinghouse can receiveemergency data from multiple emergency data sources 262 for a singleemergency. For example, in some embodiments, when a person calls 9-1-1from a mobile phone, the clearinghouse can receive a location from themobile phone (such as through the location ingestion module 271) and aheartrate from a smartwatch that the person is wearing (such as throughadditional data ingestion module 272). Or for example, in someembodiments, when a person calls 9-1-1 from a mobile phone, theclearinghouse can receive a location from the mobile phone and medicalinformation associated with the person from a third-party server 261.

The set of ingestion modules 278 optionally include a location ingestionmodule 271, an additional data ingestion module 272, and one or moreother data ingestion modules 273. In some embodiments, the locationingestion module 271 is an emergency location service ingestioninterface for posting or receiving emergency locations. In someembodiments, the location ingestion module 271 is a REST API thatreceives an HTTP POST including location data when an emergency alert isgenerated (e.g., when an emergency call is made from a cell phone). Thelocation data includes a location generated concurrently or in responseto the generation of the emergency alert. In some embodiments, thelocation data includes a location generated before the emergency alert.For example, when an emergency call is made from a cell phone, therebygenerating an emergency alert, the location ingestion module 271receives a location recently generated by the phone but before theemergency alert was generated, ensuring that a location for theemergency is available as quickly as possible. In some embodiments, thelocation data includes a device-based hybrid location generated by anelectronic device 210 that generated the emergency alert. In someembodiments, the location data includes a location generated by a secondelectronic device communicatively coupled to the electronic device thatgenerated the emergency alert. The location ingestion module 271 isintegrated into an electronic device 210 through a mobile applicationinstalled on the device 210 or integrated into the firmware or operatingsystem of the electronic device 210.

In some embodiments, the location data is generated by the electronicdevice 210 before the emergency and is accessible to a PSAP during anemergency. For example, a taxi company may have software that transmitsthe location of its cars or assets to the emergency clearinghouse 270preemptively. Thus, when an emergency arises, the location of theaffected taxi can be made accessible quicker to send help. In someembodiments, the location data is generated by the electronic device 210after the emergency has commenced and is made accessible to a PSAPduring the on-going emergency. For example, updated location data of ahijacked taxi is also periodically transmitted to the emergencyclearinghouse 270 and made accessible to a PSAP.

In some embodiments, the additional data ingestion module 272 is aninterface for posting or receiving static or dynamic emergency profiledata (hereinafter, “additional data” or “additional information”). Insome embodiments, additional data comprises medical data, personal data,demographic data, health data, or any combination thereof. Examples ofmedical data include information relating to a person's medical history,such as past surgeries or preexisting conditions. Examples of personaldata include a person's name, date of birth, height, weight, occupation,address(es) (e.g., home address, work address, etc.), spoken languages,and other personal information. Examples of demographic data include aperson's gender, ethnicity, age, etc. Examples of health data includeinformation such as a person's blood type or heartrate. In someembodiments, additional data comprises data received from connecteddevices such as vehicles, IoT devices, and wearable devices. Forexample, some intelligent vehicle systems generate and send dataregarding a crash, such as the speed at which the vehicle was movingjust before the collision, where the vehicle was struck, the number ofoccupants, etc. In some embodiments, the additional data ingestionmodule 272 is a REST API (e.g., a JSON (JavaScript Object Notation) RESTAPI). For example, in some embodiments, when an emergency call is madefrom a cell phone, thereby generating an emergency alert, the cell phonereceives a heartrate of the person who made the emergency call from asmartwatch worn by the person and communicatively coupled to the cellphone (e.g., Wi-Fi or Bluetooth connectivity). The cell phone sends theheartrate to the additional data ingestion module 272, along with anyother additional data, in an HTTP POST. In some embodiments, theadditional data ingestion module 272 is integrated into an electronicdevice 210 through a mobile application installed on the device 210 orintegrated into the firmware or operating system of the electronicdevice 210. In some embodiments, additional data is sent to theadditional data ingestion module 272 from a network server. Theadditional data ingestion module 272 is accessed by any connectedplatform that receives data that might be relevant in an emergency.Connected platforms optionally send additional data to the additionaldata ingestion module 272 at any time. For example, in some embodiments,a website, web application, or mobile application integrated with theadditional data ingestion module 272 that allows users to createprofiles sends additional data included in the profiles to theadditional data ingestion module 272 every time a profile is created orupdated.

In some embodiments, the set of ingestion modules 278 includes one ormore other data ingestion modules 273. Another data ingestion module 273is optionally an interface for posting or receiving data relevant toemergencies that is not received by the location ingestion module 271 orthe additional data ingestion module 272. In some embodiments, the otherdata ingestion module 273 receives audio or video streams during anemergency from electronic or communication devices associated with theemergency or proximal to the emergency. For example, an emergency alertis generated by an intelligent vehicle system installed in a vehicle inresponse to the vehicle experiencing a collision. In this example, theemergency alert is sent to the EMS 120 by the intelligent vehicle systemor by an electronic device communicatively coupled to the intelligentvehicle system, such as a cell phone coupled to the intelligent vehiclesystem via Bluetooth. In response to generating the emergency alert, theintelligent vehicle system additionally begins streaming audio and videofrom microphones and cameras installed inside or outside of the vehicleto the clearinghouse 270 through the other data ingestion module 273. Acell phone communicatively coupled to the intelligent vehicle systemadditionally or alternatively streams audio or video from microphonesand cameras integrated into the cell phone to the clearinghouse 270through the other data ingestion module 273. In some embodiments, theone or more other data ingestion modules 273 are REST APIs that areaccessed with an HTTP POST.

After receiving the relevant data, the set of ingestion modules 278 canstore the data in one or more clearinghouse databases 277. For example,in some embodiments, the clearinghouse databases 277 include a locationdatabase and an additional data database. In some embodiments, asdescribed above, the one or more clearinghouse databases 277 are storedon a third-party server communicatively coupled to or otherwiseaccessible by the EMS 120. In some embodiments, the set of ingestionmodules 278 tags or otherwise associates the data received by themodules with an identifier of a user or device associated with the data.For example, the set of ingestions modules 278 tag the data the receivedby the modules with a user ID number, an email address, or a phonenumber (e.g., caller ID). In some embodiments, the ingestion modules 278tag the data received by the clearinghouse 270 based on the data source(e.g., device name or type, application name, user name, phone number,corporate account, etc.).

In some embodiments, the emergency data maintained by the clearinghouseis purged. In some embodiments, the data is purged on a regular orperiodic basis. In some embodiments, data that is older than a definedthreshold is purged. In some embodiments, different data types arepurged according to different schedules and/or thresholds. For example,dynamic data (e.g., data that is subject to constant or regular change)such as location data may be more likely to become out-of-date over timeand so may be purged more frequently than static data such as apermanent home address, which may remain permanently in the databaseuntil it is replaced with an updated address.

In some embodiments, an individual or group of individuals areassociated with multiple identifiers. For example, the locationingestion module 271 receives a location generated by a phone associatedwith the phone number+1-555-555-5555, associated with John Doe. Theadditional data ingestion module 272 also receives a heartrate from asmartwatch associated with the email address johndoe@email.com, alsoassociated with John Doe. In this example, the set of ingestion modules278 tag the location with the phone number “+1-555-555-5555,” tag theheartrate with the email address “johndoe@email.com,” and associate boththe location and the heartrate with John Doe in the clearinghousedatabases 277.

In some embodiments, as depicted in FIG. 2, the clearinghouse 270includes a set of retrieval modules 279. The set of retrieval modules279 optionally include a location retrieval module 274, an additionaldata retrieval module 275, and one or more other data retrieval modules276. In some embodiments, the location retrieval module 274 is aninterface for retrieving location data from the clearinghouse databases277. In some embodiments, the location retrieval module 274 is a JSONREST API that receives a query or request (e.g., in the form of an HTTPGET request) from a requesting party, such as an ESP. In someembodiments, the request is sent from a call-taking application (e.g.,call taking module 145) integrated into the ESP system 130. In someembodiments, the request (also referred to as an “emergency datarequest”) is sent from an emergency response application 260. In someembodiments, the location retrieval module 274 provides a single GETendpoint for retrieving either the latest or paginated list of locationsfor a specific caller ID (e.g., an identifier of a user or an electronicdevice associated with a user, such as a phone number). For example, asdescribed above, a phone number associated with a device 210 from whicha location was received is included in the header, body, or metadata ofthe request sent to the location retrieval module 274. The clearinghouse270 then retrieves a location or set of locations from the clearinghousedatabases 277 and delivers the location or set of locations to therequesting party. In some embodiments, the location retrieval module 274is a location information server (LIS). In some embodiments, the LIS isa NG911 standards-based XML API for the retrieval of location data fromthe clearinghouse databases 277. In some embodiments, as describedabove, the location retrieval module 274 accepts HELD requests fromrequesting parties and returns location data for a specific caller ID oranonymous reference. However, in some embodiments, the locationretrieval module 274 automatically retrieves and transmits location datausing a subscription system, as described below.

As depicted in FIG. 2, the set of retrieval modules 279 optionallyinclude an additional data retrieval module 275. In some embodiments,the additional data retrieval module 275 is a JSON REST API for theretrieval of emergency or additional data. As described above,additional data optionally includes medical data, personal data,demographic data, and health data. Additional data also optionallyincludes data received from connected devices such as vehicles, IoTdevices, and wearable devices. In some embodiments, the additional dataretrieval module 275 receives a query or request (e.g., in the form ofan HTTP GET request) from a requesting party, such as an ESP. In someembodiments, the request (also referred to as an “emergency datarequest”) is sent from an emergency response application 280. Theadditional data then retrieves additional data associated with aspecific or particular identifier of a user or an electronic deviceassociated with the user, such as a phone number, and returns the datato the requesting party. In some embodiments, the set of retrievalmodules 279 further includes one or more other data retrieval modules276, which function similarly to the location retrieval module 274 andadditional data retrieval module 275, for the retrieval of data storedin the clearinghouse databases 277 not retrieved by the locationretrieval module 274 or additional data retrieval module 275. However,in some embodiments, the additional data retrieval module 275automatically retrieves and transmits additional data using asubscription system, as described below.

In some embodiments, a retrieval module within the set of retrievalmodules 279 and a corresponding ingestion module within the set ofingestion modules 278 form a sub-clearinghouse. For example, in someembodiments, location ingestion module 271 and location retrieval module274 combine to form location clearinghouse 170A (as shown in FIG. 1B).Likewise, in some embodiments, additional data ingestion module 272 andadditional data retrieval module 275 combine to form additional dataclearinghouse 170B. In some embodiments, a requesting party is onlygiven access to a particular sub-clearinghouse. For example, a policeofficer is only given access to the location clearinghouse 170A, whilean EMT (emergency medical technician) is only given access to theadditional data clearinghouse 170B. However, a requesting party is givendifferential access to the clearinghouse 170, sub-clearinghouses, orparticular emergency data categories within the clearinghouse 170 basedon any factor or set of factors. In some embodiments, a requesting partyinitiates a query or request (e.g., an emergency data request) using anemergency response application 260 (as described below), which in turngenerates the query and transmits the query to the clearinghouse 270.

Emergency Data Subscription System

As described above, in some embodiments, an emergency management system(EMS) maintains a clearinghouse 270 that obtains and shares emergencydata to aid emergency service providers (ESPs) in responding toemergencies. During an emergency, in some embodiments, an ESP can sendan emergency data request to the EMS through the emergency responseapplication 260, and, in response, the EMS can send any availableemergency data associated with the emergency back to the emergencyresponse application 280. In some embodiments, as described above, theemergency response application 280 includes an identifier associatedwith an emergency alert in the emergency data request. The EMS can thenuse the identifier associated with the emergency alert to retrieveemergency data associated with the emergency alert from theclearinghouse. For example, as described above, an ESP 230 (e.g., apublic safety answering point (PSAP)) can receive an emergency alert inthe form of a 9-1-1 phone call (representative of an emergency orpotential emergency) from a mobile phone associated with a phone number(e.g., (555) 555-5555). The ESP 230 can then send an emergency datarequest including the phone number (e.g., the identifier of theemergency alert) to the EMS, which can then retrieve any emergency datawithin the clearinghouse associated with the phone number and return theavailable emergency data to the requesting ESP 230. This process ofreturning emergency data to the emergency response application 280 inresponse to an emergency data request is referred to as “pulling”emergency data from the clearinghouse.

However, in some embodiments, the EMS can “push” emergency data from theclearinghouse 270 to the emergency response application (e.g., the EMScan send emergency data to the emergency response application 280without receiving an emergency data request). In some embodiments, theEMS pushes emergency data to the emergency response application 280using an emergency data subscription system. Using the emergency datasubscription, a recipient (or potential recipient) of emergency datafrom the clearinghouse 270 can subscribe to the clearinghouse 270 for aparticular device identifier, user identifier, or ESP account(hereinafter, “subscription”). After subscribing to a subscription, therecipient (e.g., an ESP) may automatically receive updates regarding thesubscription without first sending an emergency data request. Forexample, in some embodiments, if an ESP subscribes to a phone number,whenever the clearinghouse 270 receives updated emergency dataassociated with the phone number, the clearinghouse 270 canautomatically send the updated emergency data associated with the phonenumber to the ESP (e.g., through the emergency response application280), without first receiving an emergency data request including thephone number. For example, in some embodiments, if a recipient issubscribed to a particular phone number, and the clearinghouse 270receives a new or updated location associated with the particular phonenumber, the clearinghouse 270 will instantly and automatically push thenew or updated location associated with the particular phone number tothe recipient the moment that the new or updated location is received bythe clearinghouse 270, without the recipient having to send an emergencydata request. In some embodiments, when an ESP or ESP personnel accessesthe emergency response application 280 at a computing device associatedwith the ESP or ESP personnel, the EMS establishes a websocketconnection with the computing device in order to push emergency dataregarding a subscription to which the ESP or ESP personnel is subscribedto the emergency response application 280. Web Socket is a type ofcomputer communications protocol. A websocket connection is alongstanding internet connection between a client and a server thatallows for bidirectional communication between the client and serverwithout the client needing to send data requests to the server, whichdifferentiates the Web Socket computer communications protocol fromother types of computer communications protocols such as theHyperTextual Transfer Protocol (HTTP). The WebSocket protocol is oftenused by chat clients to facilitate user to user webchats. In someembodiments, the EMS establishes a websocket connection with a computingdevice (e.g., an ESP console 130) in response to receiving an emergencydata request. In some embodiments, the EMS establishes a websocketconnection with an ESP console when an ESP personnel logs into theemergency response application 280 at the ESP console. In someembodiments, the EMS establishes a websocket connection with a responderdevice when an ESP personnel logs into the emergency responseapplication 280 at the responder device. In some embodiments a websocketconnection established between the EMS and a computing device associatedwith ESP personnel is maintained by the EMS for the duration of the ESPpersonnel's log-in session.

In some embodiments, the EMS automatically subscribes a recipient to asubscription (e.g., a particular device identifier or user identifier)in response to receiving an emergency data request including thesubscription or an identifier of the subscription. For example, in someembodiments, when an ESP personnel sends an emergency data requestincluding a phone number to the EMS through their ESP console (e.g.,through the emergency response application 280), the EMS subscribes theESP personnel to the phone number and establishes a websocket connectionwith the ESP console. Then, whenever the clearinghouse 270 receivesupdated emergency data associated with the phone number, the EMS canautomatically push the updated emergency data associated with the phonenumber to the ESP console. For example, an ESP personnel logs into anemergency response application 280 in communication with the EMS on theESP personnel's ESP console. Subsequently, the ESP personnel receives a9-1-1 call from a mobile phone and then generates and sends an emergencydata request including the phone number of the mobile phone to the EMSthrough the emergency response application 280. The EMS then uses thephone number of the mobile phone to retrieve the most recent locationassociated with the mobile phone received by the clearinghouse andreturns the most recent location associated with the mobile phone to theESP personnel through the emergency response application 280. The EMSsimultaneously subscribes the ESP personnel to the phone number of themobile phone and establishes a websocket connection between the EMS andthe ESP console and automatically pushes any updated emergency data(e.g., locations) associated with the phone number received by theclearinghouse to the emergency response application 280 as soon as theupdated emergency data associated with the phone number is received bythe clearinghouse 270.

In some embodiments, an ESP is associated with a unique ESP account IDthat an ESP or ESP personnel can subscribe to. The EMS can thenestablish a websocket connection with a computing device associated withan ESP or ESP personnel subscribed to the unique ESP account ID and pushemergency data associated with the unique ESP account ID to thecomputing device (e.g., through the emergency response application 280)whenever new or updated emergency data associated with the unique ESPaccount ID is received by the clearinghouse 270. For example, in someembodiments, when the clearinghouse 270 receives a location associatedwith an emergency alert (e.g., when a person calls 9-1-1 from a mobilephone and the mobile phone responsively sends a current location of themobile phone to the clearinghouse 270), the EMS retrieves one or moregeofences (as described below) associated with each ESP registered withthe EMS and determines which (if any) of the geofences that the locationassociated with the emergency alert falls within. The EMS then tags thelocation associated with the emergency alert with the unique ESP accountIDs associated with each of the ESPs associated with geofences that thelocation associated with the emergency alert falls within. For example,if four ESPs are registered with the EMS—ESP A, ESP B, ESP C, and ESPD—and the clearinghouse 270 receives a location associated with anemergency that falls within the one or more of the geofences associatedwith ESP A and ESP D, the EMS can tag the location associated with theemergency alert with the unique ESP account ID associated with ESP A andthe unique ESP account ID associated with ESP D. The EMS can then pushthe location associated with the emergency alert to any ESPs or ESPpersonnel with an established websocket connection with the EMS andcurrently subscribed to either the unique ESP account ID for ESP A orthe unique ESP account ID for ESP D. In some embodiments, when an ESPpersonnel logs into the emergency response application 280, acommunication is sent to the EMS that includes one or more unique ESPaccount IDs that the ESP personnel or their respective ESP is currentlysubscribed to.

Emergency Data Geofencing

FIG. 3 depicts a diagram of a geofence applied to a clearinghouse foremergency data. In some embodiments, a geofence module 362 is applied tothe clearinghouse 370 to protect potentially sensitive emergency datausing geospatial analysis. In some embodiments, as described above withrespect to FIG. 2, the clearinghouse 370 includes a set of ingestionmodules 378 and a set of retrieval modules 379. The set of ingestionmodules can receive emergency data, or other information that can beuseful in responding to an emergency, from a variety of sources. Forexample, in some embodiments, a smartphone sends emergency data to theclearinghouse 370 in the form of an HTTP POST API call in response to auser of the smartphone initiating a 911 emergency call. As depicted inFIG. 3, in some embodiments, when emergency data (e.g., an emergencylocation or additional emergency data) is sent from an electronic device310 to the clearinghouse 370, the emergency data is first processed by ageofence module 362 before being received by the set of ingestionmodules 378 within the clearinghouse 370. Similarly, in someembodiments, when an emergency data request is sent from a requestingparty (e.g., the emergency response application 380, as describedbelow), the emergency data request is processed by the geofence module362 before being received by the set of retrieval modules 379 fordisplay on a GUI of the emergency response application 380 on acomputing device of the requesting party.

In some embodiments, as mentioned above, a geofence module 362 isapplied to the clearinghouse 370 to protect potentially sensitiveemergency data using geofences. Generally, a geofence is a virtualperimeter for a real-world geographic area. A geofence can bedynamically generated—as in a radius around a point location—or ageofence can be a predefined set of boundaries (such as school zones orneighborhood boundaries). The use of a geofence is called geofencing,and one example of usage involves a location-aware device of alocation-based service (LBS) user entering or exiting a geofence. Entryor exit from a geofence could trigger an alert to the device's user aswell as messaging to the geofence operator. The geofence information,which could contain the location of the device, could be sent to amobile telephone or an email account.

For emergency response, an emergency service provider (public or privateentities) may be given jurisdictional authority to a certaingeographical region or jurisdiction (also referred to as “authoritativeregions”). In the context of emergency services, one or more geofencesmay correspond to the authoritative region of an ESP. In many cases, theESP is a public entity such as a public safety answering point (PSAP) ora public safety service (PSS; e.g., a police department, a firedepartment, a federal disaster management agency, national highwaypolice, etc.), which have jurisdiction over a designated area(sometimes, overlapping areas). Geofences are used to define thejurisdictional authority by various methods and in various GeographicInformation System (GIS) formats. In some embodiments, geofences onlyrepresent authoritative regions if the geofence has been assigned orverified by a local, state, or federal government. In some embodiments,geofences represent assigned jurisdictions that are not necessarilyauthoritative regions. For example, in some embodiments, a geofence isunilaterally created by its associated ESP without verification orassignment by a local, state, or federal government.

Geofences can be defined in various ways. For example, in someembodiments, a geofence comprises one or more of the following: a countyboundary, a state boundary, a collection of postal/zip codes, acollection of cell sectors, simple shapes, complex polygons, or othershapes or areas. In some embodiments, geofences comprise approximationswhere the “approximated” geofence encloses an approximation of theauthoritative region.

Updates to geofences may be required over time because the authoritativeregions may change over time. Geofences may change over time (e.g., anew sub-division has cropped up) and require updates. In someembodiments, the systems and methods described herein allow geofences tobe updated (e.g., a PSAP administrator can upload updated geofence GISshapefiles).

For maintaining the privacy, security and integrity of the data,geofencing may be applied to emergency data. For example, applyinggeofence filters to the emergency data allows additional avenues formonitoring, both visibility and control, over the clearinghouse todetect anomalies/spikes and reduce the risk of security breaches.

In some embodiments, the emergency data is obtained from an emergencydata source 364 (such as an electronic device or third-party server, asdescribed above). On the retrieval side, in some embodiments, anemergency data recipient 380 accesses the clearinghouse 370 by sendingan emergency data request to the clearinghouse 370, as described above.An ingestion geofence 366 (also referred to as “upstream filtering”) isapplied to restrict sending of data from emergency data sources 364 tothe clearinghouse 370 from geographical areas that are not covered bythe “combined authoritative jurisdiction” (e.g., covered one or moreprovisioned geofences in the geofence database (not shown)). In someembodiments, the ingestion geofence (also referred to as an “ingressfilter”) is applied to the ingestion module 378 to protect againstaccidental breaches of privacy. In some embodiments, the ingestionmodule 378 of the clearinghouse 370 drops location payloads that do fallwithin the geographical region covered by the “combined authoritativeregion.”

In some embodiments, the clearinghouse 370 comprises one or moredatabases 377 (e.g., a database storing emergency data). For example, insome embodiments, the retrieval module 379 obtains emergency data from aclearinghouse database 377 to send to an emergency data recipient 380(e.g., an ESP) in response to an emergency data request, as describedabove. In some embodiments, the retrieval geofence 372 (also referred toas an “egress filter”) is applied at the retrieval module 379 of theclearinghouse 370. Applying geofencing to retrieved emergency data willprotect against abuse and limit the scope of security breaches in caseswhere credentials have been compromised. In some embodiments, one ormore geofences are associated with one or more credentials associatedwith an ESP agency or organization. In some embodiments, the credentialsassociated with an ESP agency or organization confers authorization toaccess data such as emergency data from the clearinghouse. In someembodiments, specific authorization to access data is grantedindividually to members of a PSAP through tokens derived from thecredentials for that PSAP.

In some embodiments, when the retrieval module 379 checks thecoordinates of current location data (within retrieved emergency data)associated with a device identifier with the geofence(s) associated withthe credentials in an emergency data request. If the current location iswithin the geofence region (enclosed by the geofence(s)), the currentlocation is returned to the ESP and displayed within the ESP console. Ifnot, the module 379 will return a “not found” message (as opposed to theretrieved location is outside the geofence) to protect privacy.

In some embodiments, geofences can be used for reporting results forinternal metrics and monitoring the system. For example, the number ofemergency data requests, locations provided, “no location found” etc.,can be obtained for a geofence(s) associated with a PSAP. Using singleor combined geofences, the emergency data can be obtained oncounty-wide, city-wide, postal code, course grid (rectangle overlay),state-wide, or country-wide basis. In some embodiments, ingress andegress counters (e.g., percent of emergency sessions where the locationdata was received, but not queried) and other similar metrics can becalculated and analyzed to identify problems and spikes. In someembodiments, different geofences are used for retrieval and forreporting.

In some embodiments, a given incident (e.g., an incident associatedemergency alert, as described below) can be determined to fall within aplurality of geofences, as described below. In some embodiments,emergency data for the incident is pushed to each PSAP having a geofencethat the incident (e.g., a location associated with the incident) fallswithin. In some embodiments, emergency data for the incident is pushedto a subset of PSAPs having a geofence that encloses or encompasses theincident. In some embodiments, the location data of an individual deviceidentifier is not pushed to more than one PSAP at one time. In someembodiments, wherein a device identifier egresses a geofence in whichcommunication began and ingresses into a neighboring geofence, thelocation data is autocratically pushed to the neighboring PSAP withjurisdiction over the ingress geofence.

To determine the appropriate ESP(s) for sharing emergency data, theauthoritative jurisdiction (defined by one or more geofences) of an ESP(e.g. primary agency) has to be evaluated. In case of irregularities(e.g. overlaps, islands, or other irregular features), steps may betaken to check with respective agency, geographical boundaries (nationaland international borders, county lines, rivers, hills, etc.), or otherauthority. In some embodiments, call routing data may be analyzed to seewhich ESP is answering the emergency call.

Raw geofences may be pre-processed to generate processed geofences usinga variety of techniques. For removing irregularities, a geofence may beprocessed to resolve overlaps, remove islands and projections, smoothboundaries, modifying the file format or size, etc.

In some cases, there may be overlap between geofence of two or moreESPs. In some embodiments, the emergency data may be shared with the twoor more ESPs to err on the side of making mission critical informationto all entities that may be involved in the emergency response. In someembodiments, the two or more ESPs are primary agencies (e.g. PSAPs) andthe emergency data has to be shared with one appropriate ESP. Todetermine the appropriate ESP(s) for sharing emergency data, theauthoritative jurisdiction (defined by one or more geofences) of theoverlapping ESPs by checking with respective agency, geographicalboundaries (national and international borders, county lines, rivers,hills, etc.), sample routing data, etc. In contrast, if the overlappingESPs include one or more secondary ESPs, the overlap may be retained andemergency data may be shared with one or more ESPs (e.g. one primaryagency and two secondary agencies).

In some embodiments, a buffer (e.g., +10 km) is added to the geofence(s)so that results within the buffer zone are also returned. In many cases,PSAPs have discretion and incentive to respond to emergencies that arebeyond their authoritative jurisdiction. As an example, a geofence thatis a circular area with a radius of 10 km would have an area of 100 π or˜314 km2, whereas the same area with a 10 km buffer around itscircumference would have yield a combined radius of 20 km and a combinedarea of 400 π or ˜1256 km2. In some embodiments, the buffer is from 0.5km to 5 km, from 0.5 km to 10 km, from 0.5 km to 15 km, from 0.5 km to20 km, from 0.5 km to 25 km, or from 0.5 km to 30 km. In someembodiments, the buffer is from 1 km to 5 km, from 1 km to 10 km, from 1km to 15 km, from 1 km to 20 km, or from 1 km to 30 km. In someembodiments, the buffer is at least 0.1 km, at least 0.2 km, at least0.3 km, at least 0.4 km, at least 0.5 km, at least 0.6 km, at least 0.7km, at least 0.8 km, at least 0.9 km, at least 1 km, at least 2 km, atleast 3 km, at least 4 km, at least 5 km, at least 6 km, at least 7 km,at least 8 km, at least 9 km, at least 10 km, at least 11 km, at least12 km, at least 9 km, at least 14 km, at least 15 km, at least 16 km, atleast 17 km, at least 18 km, at least 19 km, at least 20 km, at least 25km, or at least 30 km. In some embodiments, the buffer is no more than0.1 km, no more than 0.2 km, no more than 0.3 km, no more than 0.4 km,no more than 0.5 km, no more than 0.6 km, no more than 0.7 km, no morethan 0.8 km, no more than 0.9 km, no more than 1 km, no more than 2 km,no more than 3 km, no more than 4 km, no more than 5 km, no more than 6km, no more than 7 km, no more than 8 km, no more than 9 km, no morethan 10 km, no more than 11 km, no more than 12 km, no more than 9 km,no more than 14 km, no more than 15 km, no more than 16 km, no more than17 km, no more than 18 km, no more than 19 km, no more than 20 km, nomore than 25 km, or no more than 30 km.

Intelligent Multimedia System

In some embodiments, as depicted in FIG. 4, the emergency managementsystem (EMS) 420 includes an intelligent multimedia system (IMS) 450 foridentifying electronic devices (e.g., sensors) pertinent to anemergency, receiving data or multimedia from the electronic devicespertinent to the emergency, determining which of the electronic devicespertinent to the emergency are the most relevant to the emergency,transmitting data or multimedia from the electronic devices to anemergency service provider (ESP) or a member of an ESP, and managing thetransmission of the data or multimedia. In some embodiments, the IMS 450includes a sensor detection module 452, a multimedia processing module454, and a multimedia management module 456. In some embodiments, themultimedia processing module 454 includes a relevancy determinationmodule 455. As will be described in further detail, the modules includedin the IMS 450 work cooperatively to enable the functions of the IMS450.

For example, in some embodiments, an emergency alert is generated by anelectronic device, such as electronic device 510A (see FIG. 5). Forexample, a woman, Suzy, is standing in her kitchen when a fire ignitesin her oven and quickly begins spreading throughout her house. Suzy isable to escape from the kitchen and calls 9-1-1 from her cellphone,thereby initiating an emergency call. In some embodiments, in responseto the emergency call being initiated, an emergency alert is generatedby a program or application installed on the cellphone (e.g., emergencyalert program 117), and the emergency alert is transmitted to the EMS420. Alternatively, an intelligent smoke detector (e.g., a Nest ProtectSmoke and Carbon Monoxide Alarm) installed in Suzy's kitchen detects thefire and generates an emergency alert in response to detecting the fire.The intelligent smoke detector can then transmit the emergency alert tothe EMS 420. However, an emergency alert can be generated or transmittedto the EMS 420 in any other way. In some embodiments, an emergency alertincludes information such as a location, the nature of the associatedemergency, or user information about a user associated with theelectronic device from which the emergency alert was generated. In someembodiments, an emergency alert includes an identifier of the electronicdevice from which the emergency alert was generated. In someembodiments, an emergency alert includes an identifier of a userassociated with the electronic device from which the emergency alert wasgenerated. In some embodiments, the EMS detects an emergency alert whenan emergency call is made from an electronic device (e.g., a cellphone).In some embodiments, the EMS detects an emergency alert when the EMSreceives a request for multimedia from an ESP, as described below. Forexample, in some embodiments, when a person calls 9-1-1 and connectswith a 9-1-1 call taker, the call taker can generate and transmit arequest for multimedia to the EMS, which the EMS detects as an emergencyalert.

In some embodiments, after an emergency alert is detected by the EMS420, the IMS 450 can identify sensors (e.g., electronic devices410A-410C and/or 510A-510C) pertinent to the emergency associated withthe emergency alert. For example, referring back to the fire in Suzy'skitchen, after the EMS 420 receives the emergency alert associated withthe fire in the kitchen, the IMS 450 may identify both Suzy's cellphoneand the intelligent smoke detector as sensors pertinent to theemergency. In some embodiments, identifying sensors pertinent to anemergency is performed by the sensor detection module 452. In someembodiments, after identifying the sensors pertinent to the emergency(e.g., Suzy's cellphone and the intelligent smoke detector), the IMS 450can retrieve or receive data or multimedia from the sensors pertinent tothe emergency. For example, Suzy may begin recording a video of the firein her kitchen with her cellphone, and the IMS 450 can receive the videofrom the cellphone. In this example, the IMS 450 may additionally oralternatively receive data pertaining to the emergency from theintelligent smoke detector, such as the volume of smoke or type of gasesbeing produced by the fire.

In some embodiments, after receiving data or multimedia from sensorspertinent to an emergency, the IMS 450 then processes the data ormultimedia, such as through multimedia processing module 454, todetermine which of the sensors pertinent to the emergency are the mostrelevant. For example, the IMS 450 may determine that, between the datafrom the data received from the intelligent smoke detector and the videoreceived from Suzy's cellphone, the video is more relevant to theemergency. In some embodiments, in response to determining that thevideo (and therefore the cellphone) is more relevant than theintelligent smoke detector data, the EMS 420 then transmits the videoreceived from Suzy's cellphone to an ESP 430 or a member of an ESP 449.In some embodiments, after the EMS 420 transmits data or multimedia toan ESP 430 or ESP member 449, the IMS 450 can actively manage thetransmission of the data or multimedia, such as through multimediamanagement module 456, to ensure that the data or multimedia is properlyreceived by the EMS 430 or ESP member 449. For example, in someembodiments, the IMS 450 can sample the connection quality between theEMS 420 and an ESP 430 and, in response to a poor connection quality,decrease the framerate and/or resolution of a video being transmittedfrom the EMS 420 to the ESP 430. In some embodiments, the video isconverted into a series of still images when the connection is ofsufficiently poor quality.

Sensor Detection and Identification

As described throughout the present disclosure, in some embodiments,after an emergency alert is generated and transmitted to the EMS, theintelligent multimedia system (IMS) identifies sensors pertinent to anemergency associated with the emergency alert. An emergency alert may begenerated and transmitted to the EMS in a number of ways. For example,an emergency alert may be generated when a user dials an emergencynumber, such as 9-1-1 in the United States, from a cellphone. In thisexample, an emergency alert is generated by a program or applicationinstalled on the cellphone in response to the user dialing 9-1-1. Insome embodiments, the emergency alert includes additional information,such as a location generated by the cellphone (e.g., via locationservices or hybrid device-based location). In some cases, the emergencyalert is then transmitted to the EMS by the cellphone. In anotherexample, an emergency alert is generated automatically by an IoT device,such as by an intelligent smoke detector device in response to detectinga fire, as discussed above. In this embodiment, the emergency alertoptionally includes a location associated with the IoT device or alocation associated with a wireless network or another electronic devicethat the IoT device is connected to. In this embodiment, the emergencyalert is transmitted directly from the IoT device to the EMS orindirectly from another electronic device that the IoT device iscommunicatively coupled to, such a cellphone or smart home hub (e.g.,the Amazon Echo).

In some embodiments, after the EMS receives an emergency alert, the IMSidentifies a first set of sensors pertinent to an emergency associatedwith the emergency alert using a location associated with the emergencyincluded in the emergency alert. FIG. 5 depicts a map of emergencies andsensors pertinent to the emergencies. In one example, a fire 552Aignites in a room and is detected by a smart camera 510A using videoanalysis. In this example, the smart camera 510A combines color andmotion information received by the smart camera 510A and compares theinformation to predetermined models to detect a growing a fire. Inresponse to detecting the fire 552A, the smart camera 510A generates anemergency alert including a location of the smart camera, inherentlyindicative of the location of the emergency 550A. In some embodiments,the owner or user of an electronic device 510 (e.g., smart camera 510A)inputs a location, such as a physical address of a building, for theelectronic device 510 during the setup of the device. For example, ifSuzy buys a smart camera for her home, Suzy may input her home addressas the location of the smart camera while installing the device. Alocation of an electronic device 510 may be stored locally on thephysical memory of the device or stored on a network server (e.g., acloud network) associated with the device. In some embodiments, anelectronic device 510 is tagged with a hierarchy of location data. Forexample, in some embodiments, an electronic device 510 is tagged with aphysical address as well as a specific room within the physical address(e.g., Room 321, kitchen, or living room). In some embodiments, anelectronic device 510 is associated with an account of a user, such asby associating the device with a username, email address, or phonenumber of the user. In some embodiments, an electronic device 510associated with an account of a user can be tagged with differentlocations also associated with the user. For example, a user may beassociated with a home, office, and vacation home in three separatelocations. In this example, the user tags an electronic device with anyof the three separate locations. In some embodiments, an electronicdevice 510 has a location component, such as a GPS system, installed onthe device, from which the electronic device 510 can determine its ownlocation. In some embodiments, an electronic device 510 does not have alocation component but is communicatively coupled to a second electronicdevice 510 that does have a location component and is able to receive alocation to include in the emergency alert from the second electronicdevice 510. In some embodiments, after receiving an emergency alert froman electronic device 510, the EMS can determine a location of theelectronic device 510 based on an IP address of the electronic device510 or a communication network that the electronic device 510 isconnected to.

Referring again to the example of fire 552A, after smart camera 510Adetects the fire 552A, generates an emergency alert including a locationof the emergency 550A, and transmits the emergency alert to the EMS, theIMS can identify sensors pertinent to the emergency (e.g., fire 552A)using the location of the emergency 550A. In some embodiments, the EMSmaintains a sensor database 125 that stores information (e.g., location,associated users, type of device/sensor, connectivity capabilities,etc.) regarding sensors that are or have been communicatively coupledwith the EMS. In such embodiments, the IMS can identify sensorspertinent to the emergency by searching the sensor database 125 forsensors tagged with locations in the vicinity of the emergency, such assensors within a predetermined radius 554 of the location of theemergency 550A. In some embodiments, the IMS can identify sensorspertinent to the emergency using the location of the emergency 550A byidentifying a user account associated with the electronic device 510from which the emergency alert was generated and identifying additionalelectronic devices associated with the user account. The additionalelectronic devices associated with the user account may be consideredsensors pertinent to the emergency based on their association with theelectronic device 510 that generated the emergency alert. In someembodiments, the IMS can further determine locations of the additionalelectronic devices associated with the user account associated with theelectronic device 510 that generated the emergency alert. The IMS canthen compare these locations to the location of the emergency and selectthe additional electronic devices tagged with locations within apredetermined radius of the location of the emergency as sensorspertinent to the emergency associated with the emergency alert. In someembodiments, the size of the predetermined radius may be based in parton the nature of the associated emergency. For example, in someembodiments, if the nature of the emergency is a fire, devices within100 meters of the emergency may be considered pertinent; however, if thenature of the emergency is a medical emergency, only devices within 25meters of the emergency may be considered pertinent. In this example,the medical emergency may be considered a more micro event than thefire; important information regarding a medical emergency would probablybe observable on a much smaller scale than those of the fire.

In some embodiments, the EMS is able to communicate with third-partynetwork services (e.g., cloud services) that host electronic devices510. For example, a Nest Cam IQ (a smart camera device) may be hosted ona cloud network owned and managed by Nest, and an Amazon Echo (a smarthome hub device) may be hosted on a cloud network owned and managed onAmazon. Or, for example, both the Nest Cam IQ and the Amazon Echo may behosted on Amazon's cloud network. In such embodiments, the EMS may beable to communicate with the third-party networks to identify electronicdevices hosted or managed by the third-party networks and tagged withlocations in the vicinity of an emergency, such as by sending a query tothe third-party networks including the location of the emergency. Insuch embodiments, a third-party network can respond to the query byreturning a list of devices in the vicinity of the emergency andinformation about the devices on the list (e.g., location, associatedusers, type of device/sensor, connectivity capabilities, etc.). In someembodiments, when two or more electronic devices 510 are jointly ownedor manufactured by the same producer, the devices will automaticallysync in response to an emergency alert generated by one of the devices.For example, in some embodiments, when a Nest Protect smoke and carbonmonoxide detector detects a fire and generates an emergency alert, ajointly owned Nest Cam will automatically activate and begin recordingvideo to capture any possible causes or reactions of the fire. In suchan embodiment, information regarding the Nest Cam may be included in theemergency alert generated by the Nest Protect.

In some embodiments, the IMS can identify sensors pertinent to anemergency by identifying a communication network that the electronicdevice 510 that generated the emergency alert associated with theemergency is connected to. The IMS can then detect additional electronicdevices that are communicatively coupled to the same communicationnetwork and identify those additional electronic devices as sensorspertinent to the emergency. For example, in the case of fire 552A, theIMS may that cellphone 510C is connected to the same Wi-Fi network assmart camera 410 a. In some embodiments, the EMS or IMS can identifysensors pertinent to an emergency by prompting the electronic device 510that generated the emergency alert to broadcast a detection signal toadditional electronic devices in the vicinity of the electronic device510 that generated the emergency alert. Any additional electronicdevices that are capable of receiving the broadcast signal may then senda signal back to the electronic device 510 that generated the emergencyalert indicating that they are in the vicinity. The electronic device510 that generated the emergency alert may then relay this informationto the EMS or IMS. Alternatively, after receiving the detection signal,an additional electronic device may send a signal directly to the EMS orIMS indicating that it is in the vicinity of the emergency. In someembodiments, the IMS identifies the electronic device that generated anemergency alert as a sensor pertinent to the emergency associated withthe emergency alert by default. In some embodiments, the IMS identifiesonly active (e.g., powered on) electronic devices as sensors pertinentto an emergency. However, the IMS can identify sensors pertinent to anemergency in any other way.

In the example of fire 552A, as depicted in FIG. 5, the IMS detects twoother electronic devices within a predetermined radius of the locationof the emergency 550A550A, smart thermometer 510B and cellphone 510C,and identifies them as sensors pertinent to the emergency. The IMSadditionally identifies the smart camera 510A as a sensor pertinent tothe emergency, since the smart camera 510A generated the emergencyalert. In this example, although there are four additional electronicdevices in the area surrounding the location of the emergency 550A550A,but they are located outside of the predetermined radius 554 and aretherefore not identified as pertinent to the emergency. In this example,the IMS additionally identifies three additional electronic devices (thethree devices in the area labeled A) that are associated with the sameuser account as the smart camera 510A that generated the emergency alertassociated with the fire 552A. However, these devices are located farfrom the location of the emergency 550A550A and are likewise notidentified by the IMS as sensors pertinent to the emergency.

In a second emergency, medical emergency 552B, as depicted in FIG. 5,William has fallen in his apartment and broken his leg. In this example,William gives a voice command to his smart speaker 510D (e.g., an AmazonEcho) to contact an emergency service provider, such as by saying“Alexa, call 9-1-1.” In response, the smart speaker 510D initiates anemergency call, generates an emergency alert, and transmits theemergency alert to the EMS. In this example, the IMS identifies twoadditional electronic devices associated with William's user account,smart camera 510E and smart thermometer 510F. Because of their mutualaffiliation with William, smart camera 510E and smart thermometer 510Fare identified by the IMS as sensors pertinent to medical emergency552B.

Gathering Multimedia

In some embodiments, after receiving an emergency alert and identifyingsensors pertinent to the associated emergency, the EMS or IMS canreceive data or multimedia (also referred to as “multimedia contents”)from the set of sensors identified by the IMS as pertinent to theemergency. For example, referring again to the fire 552A depicted inFIG. 5, after identifying the smart camera 510A, the smart thermometer510B, and the cellphone 510C as the set of sensors pertinent to theemergency, the EMS or IMS can receive data or multimedia from each ofthe sensors within the set of sensors pertinent to the emergency. Forexample, the EMS or IMS may receive a video recording or video feed fromthe smart camera 510A, temperature data from the smart thermometer 510B,and an audio recording or audio feed from the cellphone 510C.Alternatively, in some embodiments, the EMS or IMS receives a secondvideo recording or video feed from the cellphone 510C. Referring now tothe medical emergency 552B depicted in FIG. 5, after identifying thesmart speaker 510D, the smart camera 510E, and the smart thermometer510F as the set of sensors pertinent to the emergency, the EMS or IMScan receive data or multimedia from each of the sensors within the setof sensors pertinent to the emergency. In this example, the EMS or IMSreceives an audio recording or audio feed from the smart speaker 510D, avideo recording or video feed from the smart camera 510E, andtemperature data from the smart thermometer 510F. In some embodiments,the EMS or IMS is configured to receive any type of data or multimediafrom an electronic device. In some embodiments, data or multimedia issent from an electronic device to the EMS through end-to-end encryption.

In some embodiments, the EMS or IMS receives data or multimedia from anelectronic device by requesting data from the electronic device. Forexample, in some embodiments, after identifying an electronic device asa sensor pertinent to an emergency, the EMS or IMS can send a query(e.g., an HTTP GET request) to the electronic device or to a serverhosting the electronic device. In some embodiments, the query includescredentials or accessing keys for verifying the EMS. In someembodiments, after receiving the query from the EMS, the electronicdevice or the server hosting the electronic device can transmit datagathered by the electronic device to the EMS or IMS. In someembodiments, after receiving the query from the EMS, the electronicdevice or the server hosting the electronic device verifies thecredentials or access keys of the EMS before transmitting data gatheredby the electronic device to the EMS.

In some embodiments, the EMS or IMS receives or obtains data ormultimedia from an electronic device without sending a query. Forexample, in some embodiments, after an electronic device generates anemergency alert, the electronic device automatically transmits data ormultimedia gathered by the electronic device to the EMS, such as bysending an HTTP POST to the EMS. In some embodiments, after the IMSidentifies an electronic device as a sensor pertinent to an emergency,the IMS can send a signal, either directly to the electronic device orindirectly to the electronic device through a second electronic devicecommunicatively coupled to the electronic device or through a serverhosting the electronic device, indicating that the electronic device hasbeen identified by the IMS as a sensor pertinent to the emergency. Inresponse to receiving the signal, the electronic device canautomatically transmit data or multimedia gathered by the electronicdevice to the EMS, such as by sending an HTTP POST to the EMS.

In some embodiments, the EMS is configured to wake an electronic devicethat the IMS has identified as a sensor pertinent to an emergency toreceive data or multimedia from the electronic device. For example,referring again to the medical emergency 552B depicted in FIG. 5, if thesmart camera 510E is in an inactive state (e.g., partially or completelypowered off), after receiving the emergency alert from the smart speaker510D and identifying the smart camera 510E as a sensor pertinent to themedical emergency 552B, the EMS can send a signal directly or indirectlyto the smart camera 510E (such as through smart speaker, if the smartspeaker 510D is communicatively coupled to the smart camera 510E, orthrough a third-party server hosting the smart camera 510E) promptingthe smart camera 510E to enter an active state (e.g., powered on) andbegin recording pictures or video. The smart camera 510E can thentransmit the pictures or video to the EMS.

The EMS or IMS can receive data or multimedia from an electronic devicein various ways. In some embodiments, the EMS receives a video streamfrom an electronic device (such as a cellphone or smart camera) directlyfrom the electronic device. For example, in some embodiments, the EMSprovides a streaming endpoint for the electronic device to transmit thevideo stream to. In some embodiments, the EMS receives a video streamfrom an electronic device by accessing a website provided by athird-party server or network that hosts the video stream from theelectronic device. In some embodiments, the EMS receives a video streamby repeatedly querying an electronic device that is recording a video(such as a smart camera) for a still image. For example, in someembodiments, the EMS receives a video stream from a smart camera byquerying the smart camera for an image once a second (or twice a second,etc.) for every second that the video stream is being recorded by thesmart camera. The EMS can then patch the sequential still imagestogether to recreate the video stream.

Data and Multimedia Processing and Relevancy Determination

After receiving an emergency alert, identifying a set of sensorspertinent to an emergency associated with the emergency alert, andreceiving data or multimedia from the sensors pertinent to theemergency, the IMS can process and determine the relevancy of the dataor multimedia received from the sensors pertinent to the emergency invarious ways. Because emergency service providers operate in highpressure situations with lives at risk, they must be able to processinformation quickly and efficiently. Emergency service providers alsoneed to be provided with only relevant information, as many emergencysituations do not allow emergency service providers the time to siftthrough data to find the relevant information. In some embodiments, theIMS processes and determines the relevancy of data or multimediareceived from sensors pertinent to an emergency so that the EMS canprovide emergency service providers with only the most relevant data ormultimedia.

In some embodiments, after receiving data or multimedia from sensorspertinent to an emergency, the IMS processes the data or multimedia(such as through multimedia processing module 454) for audio or visualcues. For example, in some embodiments, if the IMS receives an audiorecording or stream, the multimedia processing module 454 can analyzethe audio for key sounds, words, voices, etc. For example, in someembodiments, the multimedia processing module 454 can use speechrecognition techniques to detect key words such as “emergency,”“weapon,” “fire,” “help,” and the like. If the multimedia processingmodule 454 detects a key word in the audio received from a particularsensor pertinent to an emergency, the IMS may select or identify theparticular sensor as a relevant sensor. In some embodiments, themultimedia processing module 454 includes a list of predetermined keywords. In some embodiments, the multimedia processing module 454 candetect a volume level (e.g., a decibel level) of an audio recording orstream received from a particular sensor pertinent to an emergency. Ifthe multimedia processing module 454 detects a volume level that exceedsa threshold volume level, the IMS may select or identify the particularsensor as a relevant sensor. In some embodiments, the threshold volumelevel is a static value. In some embodiments, the threshold volume levelis a predetermined value. In some embodiments, the multimedia processingmodule 454 compares the volume level to an average volume level of audioreceived from the particular sensor and the IMS may selects oridentifies the particular sensor as a relevant sensor if the volumelevel exceeds the average volume level.

In some embodiments, if the IMS receives an audio recording or streamfrom a particular sensor pertinent to an emergency, the multimediaprocessing module 454 can use speech recognition techniques totranscribe the audio into a text transcription. FIG. 6 depicts twoexamples of text transcriptions of audio received from a sensorpertinent to an emergency by the IMS. In the first example, the texttranscription (lines 656A-656B) is a transcription of an audio streamreceived from a recording device installed in a bank lobby. An armedrobbery begins in the bank lobby and one of the bank's tellers presses apanic button under his desk, thereby generating an emergency alert thatis subsequently transmitted to the EMS. In response to receiving theemergency alert, the IMS identifies sensors pertinent to the emergency(e.g., the armed robbery), including the recording device installed inthe bank lobby, and begins receiving an audio stream from the recordingdevice. The multimedia processing module 454 is able to transcribe twolines of audio into text transcriptions, “Help, I've been shot and Ican't get up. Active shooter.” (line 656A) and “Officer down!” (line656B).

In the second example, the text transcription (lines 656C-656D) is atranscription of an audio stream received from a cellphonecommunicatively coupled to an intelligent vehicle system at the scene ofa car accident in which a van has struck a pedestrian in an inner-cityintersection. The pedestrian was wearing a smartwatch (e.g., an AppleWatch) with a heartrate monitor that detected, such as through theimpact of the van colliding with the pedestrian and the resulting changein the pedestrian's heartrate, that the pedestrian may be experiencingan emergency. In response, the smartwatch automatically generated anemergency alert including the location of the smartwatch (e.g., thelocation of the emergency) and transmitted the emergency alert to theEMS. In response to receiving the emergency alert, the IMS uses thelocation included in the emergency alert to identify sensors pertinentto the emergency. In this example, the IMS detects a car with anintelligent vehicle system stopped as a result of the accident near theintersection. For example, an OnStar intelligent vehicle system isintegrated into the car. The IMS, in response to receiving the emergencyalert, sends a query including the location of the emergency to theOnStar network and the OnStar network returns information regarding anyOnStar integrated vehicle in the vicinity of the emergency. In thisexample, the IMS then communicates with the OnStar network to prompt amicrophone within the car to begin recording audio and transmits theaudio to a cellphone communicatively coupled to the OnStar systemintegrated into the car. The cellphone then transmits the audio recordedby the vehicle to the EMS. The multimedia processing module is then ableto transcribe the audio into text transcriptions, “Oh my god, look, didthat man just get run over?” (line 656C), “I think he just got hit bythat van!” (line 656D), and “Someone call 9-1-1! Quick! That man needshelp!” (line 656E).

In some embodiments, after receiving audio from a sensor pertinent to anemergency and transcribing the audio into a text transcription, themultimedia processing module can then parse the text transcriptions forkey words or key terms. As depicted in FIG. 6, in the first example, themultimedia processing module identifies the following underlined wordsand terms as key words and key terms: ““Help, I've been shot and I can'tget up. Active shooter.” (line 656A) and “Officer down!” (line 656B). Inresponse to detecting key words in the text transcription of the audioreceived from the recording device, the IMS can select the recordingdevice as a relevant sensor. In the second example, the multimediaprocessing module identifies the following underlined words and terms askey words and key terms: “Oh my god, look, did that man just get runover?” (line 656C), “I think he just got hit by that van!” (line 656D),and “Someone call 9-1-1! Quick! That man needs help!” (line 656E). Inresponse to detecting key words in the text transcription of the audioreceived from the cellphone, the IMS can select the car as a relevantsensor. In some embodiments, the IMS selects a particular sensor as arelevant sensor only if the multimedia processing module identifies athreshold number of key words or key terms within the audio or within atext transcription of the audio received from the particular sensor. Insome embodiments, the multimedia processing module (or the relevancydetermination module 455) weighs different key words or key terms withdifferent amounts of relevance. For example, the key term “activeshooter” may be given a value of high relevance, the key word “help” maybe given a value of medium relevance, and the key word “officer” may begiven a value of low relevance. In some embodiments, the IMS selects aparticular sensor as a relevant sensor only if the multimedia processingmodule identifies key words or key terms, within the audio or within atext transcription of the audio received from the particular sensor,having an aggregate amount of relevance greater than a threshold amountof relevance (hereinafter, “relevance threshold”). For example, in anembodiment wherein key words or key terms are given a value of lowrelevance, medium relevance, or high relevance, the relevance thresholdmay be three or more low relevance key words/terms, two or more mediumrelevance key words/terms, or one or more high relevance keywords/terms. Or, for example, the relevance threshold may be two or morelow relevance key words/terms and one or more medium relevance keywords/terms.

In some embodiments, the multimedia processing module additionally oralternatively performs a sentiment analysis on a text transcription todetermine the sentiment of an audio recording or stream. In someembodiments, the multimedia processing module then uses the sentiment todetermine the relevancy of a particular sensor pertinent to anemergency.

In some embodiments, if the IMS receives a video recording or videostream from a particular sensor pertinent to an emergency, themultimedia processing module 454 analyzes the video for visual cues suchas key images, colors, movement/activity, animate objects, inanimateobjects, people, animals, etc. For example, in some embodiments, themultimedia processing module identifies an amount of movement (e.g.,activity or activity level) in a video, and, if the multimediaprocessing module detects activity exceeding a threshold activity level,the IMS selects the particular sensor from which the video was receivedas a relevant sensor. In some embodiments, the multimedia processingmodule compares the activity level to an average activity level receivedfrom the particular sensor and selects the particular sensor as apertinent sensor if the activity level exceeds the average activitylevel.

FIGS. 7A-7C depict embodiments of a room, such as a living room, withvarious electronic devices 710. As depicted in FIG. 7A, the room may bepopulated or occupied by any number of animate and inanimate objects,such as table 758A, dog 758B, family 758C, couch 758D, rug 758E, plant758F, and television 758G. As depicted in FIG. 7A, the room may includedifferent types of electronic devices 710, such as smart camera 710 a,smart thermometer 710B, and smart speaker 710C. Different electronicdevices 710 may have different technical capabilities and may thereforebe capable of gathering different types of information regarding anemergency. For example, as depicted in FIG. 7A, the room may experiencea fire 752. In this example, after the fire 752 ignites in the room, amember of the family 758C gives a voice command to the smart speaker710C (e.g., a Google Home device) to initiate a 9-1-1 call, such as byaudibly saying, “Hey Google, call 9-1-1.” In some embodiments, inresponse to receiving the voice command, the smart speaker 10C cangenerate an emergency alert, transmit the emergency alert to the EMS,and initiate a 9-1-1 call. In this example, after the EMS receives theemergency alert, the IMS determines that smart camera 710A and smartthermometer 710B are associated with the same user account as smartspeaker 710C and thus identifies smart camera 710A, smart thermometer710B, and smart speaker 710C as the set of sensors pertinent to theemergency (e.g., fire 752).

After identifying the smart camera 710A, smart thermometer 710B, andsmart speaker 710C as the set of sensors pertinent to the fire 752, theIMS then begins receiving data or multimedia from the three devices. Forexample, the IMS can receive an audio stream from the smart speaker710C, temperature data from the smart thermometer 710B, and a videostream from the smart camera 710A. After the IMS receives the videostream from the smart camera 710A, the multimedia processing module cananalyze the video received from the smart camera 710A. As depicted inFIG. 7B, in this example, the multimedia processing module analyzes thevideo feed and detects the following key objects: dog 758B, family 758C,and fire 752. In some embodiments, the multimedia processing moduledetects other objects in the room, such as table 758A, couch 758D, rug758E, plant 758F, and television 758G, and identifies the other objectsas not relevant to the emergency. For example, in some embodiments,pyroelectric infrared (PIR) analysis can be used to detect the presenceof humans or animals. In some embodiments, the multimedia processingmodule uses facial recognition software to identify faces of people invideos received by the IMS. The multimedia processing module canidentify faces as key objects. In some embodiments, the multimediaprocessing module uses voice or facial recognition software todistinguish between users (e.g., user identification). In response todetecting the key objects, the IMS selects the smart camera 710A (thesensor from which the video feed was received) as a relevant sensor. Insome embodiments, the IMS selects a particular sensor from a set ofpertinent sensors as a relevant sensor if a video received from theparticular sensor includes one or more key objects. In some embodiments,the IMS selects the particular sensor from the set of pertinent sensorsas a relevant sensor if the video includes a threshold rate of keyobjects per unit of time. For example, the threshold rate of key objectsper unit of time may be one or more key objects per second.Alternatively, the threshold rate of key objects per unit of time may be0.5 key objects per second. In another example, the IMS selects theparticular sensor from the set of pertinent sensors as a relevant sensorif the video includes a threshold rate of key objects per video frame.

In some embodiments, the multimedia processing module can analyze dataor multimedia received from a sensor pertinent to an emergency todetermine a likely nature of the emergency or potential threat. Forexample, as depicted in FIG. 7C, the multimedia processing can analyzevideo received from smart camera 710A to detect the fire 752, such as bycombining motion and color information received by the smart camera710A. The multimedia processing module can then identify the fire 752 asthe likely nature of the emergency. In another example, the multimediaprocessing module can detect a rapid increase in the temperature of theroom in the temperature data received from the smart thermometer 710Band determine that the likely nature of the emergency is a fire based onthe rapid increase in temperature. In some embodiments, the multimediaprocessing module can combine data and multimedia received from multiplesensors pertinent to an emergency to determine a likely nature of theemergency. For example, the multimedia processing module may combine therapid increase in temperature detected in the temperature data receivedfrom the smart thermometer 710B and the color and motion informationgathered from the video received from the smart camera 710A to determinethat the fire 752 is the likely nature of the emergency.

In some embodiments, the IMS selects one or more sensors from a set ofsensors pertinent to an emergency as relevant sensors (e.g., a set ofrelevant sensors). For example, for the fire 752, the IMS may select thesmart camera 710A and the smart speaker 710C as the set of relevantsensors for the fire 752. In another example, the IMS may select thesmart camera 710A, the smart thermometer 710B, and the smart speaker710C as the set of relevant sensors for the fire 752. The IMS may selectany number of sensors from a set of pertinent sensors as relevantsensors based on any criteria. In some embodiments, the IMS selects eachsensor within the set of sensors pertinent to an emergency, from whichdata or multimedia was received and assigned a relevancy score exceedinga predetermined relevancy score threshold, as a relevant sensor.

In some embodiments, the IMS selects a limited number of sensors from aset of sensors pertinent to an emergency as relevant sensors. Forexample, in some embodiments, the IMS selects a maximum of one relevantsensor from a set of sensors pertinent to an emergency. In someembodiments, the IMS selects a maximum of two relevant sensors from aset of sensors pertinent to an emergency. In some embodiments, therelevancy determination module 455 can calculate and assign a relevancyscore to the data or multimedia received from the sensors within the setof sensors pertinent to an emergency. For example, in some embodiments,when the multimedia processing module analyzes data or multimediareceived from a sensor pertinent to an emergency, the relevancydetermination module assigns the data or multimedia a normalizedrelevancy score between 1 and 10, wherein 10 is the highest possiblerelevancy score. In some embodiments, the relevancy determination moduledetermines a relevance score for data or multimedia received from asensor pertinent to an emergency in various ways. In some embodiments, arelevancy score is based at least in part on the type of electronicdevice from which the data or multimedia was received. For example, insome embodiments, a video stream received from a smart camera may begiven a higher relevancy score than a video stream received from acellphone, or vice versa. In some embodiments, a relevancy score isbased at least in part on the distance of the sensor from which the dataor multimedia was received from the location of the emergency. In someembodiments, the relevancy score is based at least in part on the likelynature of the emergency. For example, in some embodiments, if the likelynature of an emergency is a fire, a video stream may be given a higherrelevancy than an audio stream. In some embodiments, the relevancy scoreis based at least in part on audio or visual cues detected in the dataor multimedia, as discussed above. In some embodiments, each audio orvisual cue detected in the data or multimedia is given the same weightby the relevancy determination module. In some embodiments, differentaudio or visual cues are given different weights by the relevancydetermination module. For example, in some embodiments, a human detectedin a video received by the IMS may be given more weight by the relevancydetermination module when calculating a relevancy score for the videothan a dog detected in the video.

In some embodiments, the relevancy determination module employs machinelearning techniques when calculating a relevancy score for data ormultimedia received from a sensor pertinent to an emergency. Forexample, in some embodiments, when the relevancy determination modulecalculates and assigns a relevancy score to data or multimedia receivedfrom a sensor pertinent to an emergency, the relevancy determinationmodule stores information regarding the data or multimedia along withthe relevancy score in a database (hereinafter “relevancy scoredatabase”). In some embodiments, the information regarding the data ormultimedia stored in the relevancy score database can include sensortype, device type, timestamp, likely nature of emergency, audio orvisual cues, or any other information regarding the data or multimedia.In some embodiments, as will be described below, after identifying a setof sensors pertinent to an emergency, receiving data or multimedia fromthe set of sensors pertinent to the emergency, and determining a set ofrelevant sensors from the set of sensors pertinent to the emergency, theIMS or EMS can transmit data or multimedia from the relevant sensors toan emergency service provider (ESP). In some embodiments, the IMS canthen receive feedback on the data or multimedia from the relevantsensors from the ESP and store the feedback alongside the informationregarding the data or multimedia and the relevancy score in therelevancy score database. For example, in some embodiments, after thedata or multimedia is transmitted to the ESP, a member of the ESP mayidentify or tag the data or multimedia as relevant (e.g., positiveaffirmation), such as through feedback module 148. Conversely, themember of the ESP may tag the data or multimedia as irrelevant (e.g.,negative affirmation). In some embodiments, the member of the ESP mayrequest data or multimedia from another sensor and the IMS can interpretthis request as negative affirmation of the original data or multimediatransmitted to the ESP. However, the IMS or EMS may receive feedback(e.g., positive or negative affirmation) regarding the data ormultimedia transmitted to an ESP in any other way. In this way, overtime, the IMS can assemble a large amount of data within the relevancyscore database, which the IMS can then use to train the ability of therelevancy determination module to calculate relevancy scores. Forexample, in some embodiments, the IMS can apply a convolutional neuralnetwork to the relevancy score database to determine how much weight therelevancy determination module should give to various audio cues whendetermining the relevance of an audio stream received from a sensorpertinent to an emergency.

In some embodiments, after the IMS identifies a set of sensors pertinentto an emergency, the IMS can select one or more relevant sensors fromthe set of sensors pertinent to the emergency based on one or moreprioritization rules. Prioritization rules may be based on variousfactors, including, but not limited to: device type, media type,location, nature of emergency, or ESP capabilities. For example, in someembodiments, if the EMS is aware that an ESP that is to receive data ormultimedia is not capable of receiving a particular type of data ormultimedia, the IMS can prioritize other types of data or multimediawhen selecting relevant sensors. For example, if an ESP that is toreceive data or multimedia cannot receive or process video, the IMS canprioritize an audio stream over a video stream when selecting relevantsensors. In some embodiments, the IMS can prioritize one type of data ormultimedia over another. For example, the IMS may prioritize videostreams over audio streams when selecting relevant sensors. In someembodiments, the IMS can prioritize one type of data or multimedia overanother based on the nature of an emergency or the likely nature of theemergency. For example, if the nature of an emergency is a fire, the IMSmay prioritize video streams over audio streams. In some embodiments,the IMS can prioritize one type of data or multimedia over another basedon distance from an emergency. For example, the IMS may give priority tovideo streams within 50 meters of an emergency but prioritize audiostreams outside of 50 meters from the emergency. In some embodiments,prioritization rules are predetermined within the IMS (e.g., within therelevancy determination module). In some embodiments, prioritizationrules are received from ESPs. For example, in some embodiments, a memberof an ESP (e.g., an administrator of the ESP) can submit prioritizationrules to the EMS, such as by accessing a software program or websiteassociated with the EMS.

In some embodiments, after the IMS receives data or multimedia from asensor pertinent to an emergency, the multimedia processing module cansummarize the data or multimedia into a summarization, such as a textsummary. FIG. 8 depicts (i) an embodiment of a summarization of an audiostream received from a sensor pertinent to an emergency and (ii) anembodiment of a summarization of a video stream received from a sensorpertinent to an emergency. In the first example, the audio summary (i)is a summarization of the audio represented by lines 656C-656E depictedin FIG. 6, wherein the audio was received by the EMS from cellphonecommunicatively coupled to an intelligent vehicle system that recordedthe audio. In this example, the multimedia processing module determines,such as by analyzing the transcription depicted in FIG. 6 (as describedabove), that the incident is a possible collision involving a vehicleand a pedestrian. The multimedia processing module can use thisinformation to create a description of the audio and incorporate thedescription into the audio summary (i). In the second example, the videosummary (ii) is a summarization of the video received by the smartcamera 710A depicted in FIG. 7. In this example, the multimediaprocessing module determines, such as by analyzing the video recorded bythe smart camera 710A (as described above), that the likely nature ofthe emergency is a fire, and that there is a family and dog present inthe room. The multimedia processing module can use this information tocreate a description of the video and incorporate the description intothe video summary (ii). In some embodiments, along with a description ofthe data or multimedia received from a sensor, a summarization caninclude information such as media type, device type, location, andtimestamp. However, a summarization may include any informationregarding an emergency or a sensor. In some embodiments, the multimediaprocessing module can dynamically update summarizations of data ormultimedia received by the EMS. For example, in some embodiments, themultimedia processing module generates a new summarization of the dataor multimedia received from a sensor pertinent to an emergency everyfive or ten seconds. In some embodiments, the multimedia processingmodule generates a video summarization by selecting one or more keyframes within a video and patching the one or more key frames into avideo summarization. In some embodiments, a video summarization is asequence of still images (e.g., KeyFrames). In some embodiments, a videosummarization is a sequence of moving images (e.g., video skims). Insome embodiments, the multimedia processing module employs clusteringtechniques, which utilize fuzzy logic, to select one or more key framesfrom a cluster of frames.

In some embodiments, the IMS can select a sensor from a set of sensorspertinent to an emergency as a relevant sensor based at least in part oninput received from an ESP or a member of an ESP. An ESP may be uniquelyqualified to determine which data or multimedia is most relevant or mosthelpful in providing emergency response services. For example, in someembodiments, after the IMS identifies a set of sensors pertinent to anemergency and receives data or multimedia from the set of sensorspertinent to the emergency, the IMS generates a summarization (asdescribed above) of the data or multimedia received for each of thesensors within the set of sensors pertinent to the emergency andtransmits the summarizations to an ESP. A member of the ESP can thenread the summarizations and, using their experience and trainedintuition, select the summarizations that they believe to be mostrelevant. After receiving selection of a summarization from an ESP, theIMS can select the sensor from which the summarization was generated asa relevant sensor. In some embodiments, after the IMS identifies a setof sensors pertinent to an emergency, the EMS or IMS can generate a map941 of the sensors pertinent to the emergency and/or a list 942 of thesensors pertinent to the emergency and transmit the map 941 or list ofsensors 942 to an ESP. The map 941 or list 942 can then be displayedthrough the ESP system, such as through a desktop ESP display 931 a (asdepicted in FIG. 9A) or a mobile ESP display 931 b (as depicted in FIG.9B). The map 941 and list 942 can present, graphically and textually,respectively, information regarding the emergency and the set of sensorspertinent to the emergency. For example, the map 941 and list 942 caninclude the location of the emergency, the nature (or likely nature) ofthe emergency, the types of electronic devices, the types of data ormultimedia, the distances of the sensors from the location of theemergency, or relevancy scores of the data or multimedia. In someembodiments, a member of the ESP then selects one or more sensorsthrough the desktop or mobile ESP display 931, such as by clicking on asensor on the map 941 or list 942, and the IMS can select the one ormore sensors selected by the member of the ESP as relevant sensors. Insome embodiments, a member of an ESP can submit a radius from thelocation of an emergency through the map 941 and the IMS can select thesensors within the radius from the set of sensors pertinent to theemergency as relevant sensors.

Transmission and Management

In some embodiments, after receiving an emergency alert, identifying aset of sensors pertinent to an emergency, and selecting a set ofrelevant sensors from the set of sensors pertinent to the emergency, theEMS or IMS can transmit data or multimedia from the set of relevantsensors to an emergency service provider (ESP). For example, referringagain to the fire 752 depicted in FIGS. 7A-7C, after receiving anemergency alert generated by the smart speaker 710C in response to avoice command by the smart speaker 710C from a member of the family758C, the IMS identifies the smart camera 710A, the smart thermometer710B, and the smart speaker 710C as the set of sensors pertinent to theemergency (e.g., the fire 752), as described above. In this example, theIMS then receives data or multimedia from the smart camera 710A, thesmart thermometer 710B, and the smart speaker 710C (e.g., a first set ofmultimedia contents received from the set of sensors pertinent to theemergency), as described above, and selects the smart camera 710A as arelevant sensor (e.g., the set of relevant sensors), as described above.In this example, after selecting the smart camera 710A as a relevantsensor, the IMS then transmits the data or multimedia received from thesmart camera 710A (e.g., a second set of multimedia contents receivedfrom the set of relevant sensors) to an ESP.

In some embodiments, an ESP responding to an emergency sends a requestfor multimedia regarding the emergency to the EMS, and the EMS transmitsthe second set of multimedia contents received from the set of relevantsensors the ESP in response to receiving the request for multimedia. Forexample, an ESP may receive a 9-1-1 call regarding an emergency andsubmit a request for multimedia regarding the emergency to the EMS. Insome embodiments, the request for multimedia includes a phone number. Insome embodiments, the EMS uses a phone number included in the requestfor multimedia to locate associated emergency or a set of sensorspertinent to the emergency. In some embodiments, the request formultimedia includes a location of the emergency. In some embodiments, amember of an ESP submits a request for multimedia to the EMS through theESP system, such as by selecting a button on the ESP display. In someembodiments, the EMS identifies an ESP responsible for responding to aparticular emergency or actively responding to the particular emergencyand transmits an alert to the ESP notifying the ESP that the EMS hasaccess to electronic devices that may provide data or multimediapertinent to the emergency. In such embodiments, the EMS may present anoption (e.g., a button) to submit a request for multimedia to the ESPalongside the alert. In some embodiments, the EMS can transmit data ormultimedia received from a relevant sensor to an ESP without receiving arequest for multimedia.

In some embodiments, the EMS transmits data or multimedia received froman electronic device directly to an ESP by connecting the ESP system tothe outbound stream of data from the electronic device. For example, ifthe EMS receives a stream of temperature data from a smart thermometer,the EMS can redirect (or additionally direct) the stream of temperaturedirectly to an endpoint of the ESP system, such that the ESP systemreceives the stream of temperature data from the smart thermometerwithout the stream of data passing through the EMS first. In anotherexample, if the EMS receives a video stream from a smart camera, the EMScan direct the video stream directly to an endpoint of the ESP system.Alternatively, the EMS can provide the ESP with a weblink to a domainhosting the video stream. In some embodiments, the EMS transmits data ormultimedia received from an electronic device indirectly to an ESP(e.g., the EMS consumes the data or multimedia before relaying the dataor multimedia to the ESP). For example, if the EMS can receive a videostream from a smart camera to an endpoint of the EMS and then transmitthe video stream from the EMS to an endpoint of the ESP system.Alternatively, the EMS can generate a video stream from a smart cameraby continuously querying the smart camera for a still image (e.g., twostill images per second) and patching the images together to form avideo stream after the still images are received by the EMS. The EMS canthen transmit the patched together video stream to the ESP. In someembodiments, after receiving multimedia contents from a set of relevantsensors, the IMS can generate one or more summarizations of themultimedia contents (as described above) and transmit the one or moresummarizations to an ESP alternatively or additionally to the multimediacontents. However, in some embodiments, the EMS transmits data ormultimedia received from an electronic device to an ESP in any otherway.

In some embodiments, after transmitting data or multimedia to an ESP,the EMS actively manages the transmission of the data or multimedia tothe ESP. In some embodiments, the EMS can manage the transmission ofdata or multimedia to an ESP by dynamically changing the data ormultimedia transmitted to the ESP. FIGS. 10A and 10B depict method flowdiagrams of a method for transmitting multimedia from an electronicdevice to an emergency service provider (ESP) by an emergency managementsystem (EMS) and managing the transmission of the multimedia. In FIG.10A, the method S1000A begins with the steps of detecting an emergencyalert S1002, identifying a set of sensors pertinent to an emergencyassociated with the emergency alert S1004, receiving a first set ofmultimedia contents from the set of sensors pertinent to the emergencyS1006, selecting a first set of relevant sensors from the set of sensorspertinent to the emergency S1008, and transmitting a second set ofmultimedia contents from the first set of relevant sensors to an ESPS1010, as described above. In this example, the method continues withselecting a second set of relevant sensors from the set of sensorspertinent to the emergency S1012. In some embodiments, the ESP choosesto receive data or multimedia from a different set of sensors. Forexample, in some embodiments, a member of the ESP indicates that one ormore of the multimedia contents is not relevant (or less relevant) tothe emergency. In this example, in response to receiving the indication,the IMS can select one or more additional or alternative relevantsensors (e.g., the second set of relevant sensors) from the set ofsensors pertinent to the emergency and transmit multimedia contents fromthe second set of relevant sensors to the ESP (e.g., transmitting athird set of multimedia contents from the second set of relevant sensorsto the ESP S1014). In some embodiments, the IMS selects the second setof relevant sensors in response to determining that one or more of thefirst set of multimedia contents has become unavailable S1016, asdepicted in method S1000B in FIG. 10B. For example, during an emergency,an active sensor pertinent to the emergency may become inactive. Forexample, a user recording a video of the emergency on their cellphonemay stop recording. In another example, a fire may destroy a smartcamera recording a video of the fire or otherwise render the smartcamera incapable of recording or transmitting the video.

In some embodiments, the IMS selects the second set of relevant sensorsby selecting the next most relevant sensor(s) from the set of sensorspertinent to the emergency that were not included in the first set ofrelevant sensors. In some embodiments, the IMS selects the second set ofrelevant sensors by recalculating the relevancy score for each sensorwithin the set of sensors pertinent to the emergency. As an emergencysituation progresses, sensors pertinent to the emergency may become moreor less relevant. For example, the location of the emergency may changeduring the course of the emergency, and sensors that were closer to thelocation of the emergency may become farther than some others, renderingthem less relevant. In some embodiments, the EMS can actively monitorthe relevancy of the sensors within the set of sensors pertinent to anemergency. For example, in some embodiments, the EMS recalculates therelevancy score of each sensor within the set of sensors pertinent to anemergency every five or ten seconds. In some embodiments, the EMS canselect the second set of relevant sensors in response to determiningthat a relevancy score of a sensor within the set of sensors pertinentto the emergency or the set of relevant sensors has changed. In someembodiments, a member of an ESP can select one or more additional oralternative sensors from the set of sensors pertinent to the emergencyas relevant sensors. For example, in some embodiments, the EMS or IMSgenerates and provides an ESP with a map or list of the sensorspertinent to an emergency. In such an embodiment, after receivingmultimedia contents from a first set of relevant sensors selected fromthe set of sensors pertinent to the emergency, a member of the ESP mayselect a second set of relevant sensors from the map or list of thesensors pertinent to the emergency. However, the IMS may select a secondset of relevant sensors in any other way. In some embodiments, the IMSmay select multiple additional or alternative sets of relevant sensorsas necessary.

In some embodiments, the EMS actively manages the transmission ofmultimedia contents to an ESP by monitoring the transmission. FIG. 11depicts an embodiment of a method flow diagram of a method S1100 fortransmitting multimedia from an electronic device to an emergencyservice provider (ESP) by an emergency management system (EMS) andmanaging the transmission of the multimedia. In this embodiment, themethod S1100 begins with the steps of detecting an emergency alertS1102, identifying a set of sensors pertinent to an emergency associatedwith the emergency alert S1104, receiving a first set of multimediacontents from the set of sensors pertinent to the emergency S1106,selecting a first set of relevant sensors from the set of sensorspertinent to the emergency S1108, and transmitting a second set ofmultimedia contents from the first set of relevant sensors to an ESPS1110, as described above. In this embodiment, the method continues bysampling a connection quality of the transmission of a multimediacontent within the second set of multimedia contents to the ESP S1118.For example, the EMS can sample the bandwidth of a communication linkbetween the EMS and the ESP. Alternatively, in some embodiments, the EMSsamples the success rate of data packet transmission to the EMS. In someembodiments, in response to the connection quality falling below athreshold value, the EMS downgrades the multimedia content S1120. Forexample, in some embodiments, the EMS downgrades an audio recording oraudio stream into a text transcription (e.g., the EMS stops transmittingthe audio and begins transmitting a text transcription of the audio). Insome embodiments, the text transcription is transmitted to the ESP as anSMS or MMS text message. In some embodiments, the EMS downgrades a videorecording or video stream into still images. In some embodiments, theEMS downgrades a video stream by lowering the framerate of the videostream. In some embodiments, the EMS downgrades an audio or videorecording or stream into a summarization, as described above. However,the EMS can downgrade a multimedia content in any other way.

In some embodiments, the EMS transmits emergency data or multimedia toESPs using web real-time communication (WebRTC). WebRTC is a set ofprotocols used to provide internet browsers and mobile applications withreal-time communications capabilities using application programminginterfaces (APIs). In some embodiments, the EMS transmits emergency dataor multimedia to ESPs using the real-time transport protocol (RTP) andmanages the transmission of the emergency data or multimedia using RTPcontrol protocol (RTCP). In some embodiments, the EMS additionally oralternatively utilizes the real time streaming protocol (RTSP) whentransmitting emergency data or multimedia to ESPs. However, the EMS mayuse any system or protocol or combination of systems and protocols whentransmitting emergency data or multimedia.

Contextual Relevance

In some embodiments, an electronic device or sensor can autonomouslytransmit emergency data or multimedia to the emergency management system(EMS) when the electronic device or sensor independently determines thatit has collected relevant emergency data or multimedia. For example, insome embodiments, if a smart camera or smart thermostat autonomouslydetects a fire (such as in the example emergency illustrated by FIG.7A), the smart camera or smart thermostat can autonomously transmit anemergency alert along with any emergency data or multimedia gathered bythe device (e.g., video or temperature data; location) to the EMSwithout first being accessed by the EMS. The EMS can then determine anappropriate emergency service provider (ESP) to receive the emergencyalert and any accompanying emergency data or multimedia (such as byusing a geofence system or emergency data subscription system, asdescribed above) and transmit the emergency alert and accompanyingemergency data or multimedia to the ESP. In another example, in someembodiments, if a camera integrated into a vehicle detects a licenseplate of a wanted vehicle (e.g., a stolen vehicle or a vehicle listed inan AMBER alert), an intelligent vehicle system integrated into thevehicle can autonomously transmit an emergency alert along with anyemergency data or multimedia (e.g., photos or videos; location of thevehicle) captured by the camera to the EMS. The EMS can then determinean appropriate ESP to receive the emergency alert and transmit theemergency alert and the accompanying multimedia to the ESP. In someembodiments, the EMS can receive updated emergency data or multimediaand transmit the updated emergency data or multimedia to the ESP. Forexample, in some embodiments, an intelligent vehicle system integratedinto a private vehicle, after detecting a license plate of a vehiclelisted in an AMBER alert using a camera integrated into the privatevehicle, can transmit the location of the private vehicle to the EMS,which can then transmit the location to an ESP. The driver of theprivate vehicle can decide to follow the vehicle listed in the AMBERalert, and the intelligent vehicle system can periodically transmitupdated locations of the private vehicle to the EMS and ultimately tothe ESP. The ESP can then use the updated locations to track and findthe vehicle listed in the AMBER alert.

Privacy and Access Control

Devices capable of generating and sharing emergency data and multimediathat may be useful for emergency service providers (ESPs) and firstresponders when they are responding to emergencies have becomeplentiful. In many cases, a single person or home may possess multipledevices capable of generating and sharing emergency data and multimedia.For example, a person may have both a cell phone and a wearable device(e.g., a smartwatch) that can gather data and multimedia during anemergency. In another example, a single room within a home may contain asmart camera, a smart thermostat, and a smart speaker (such as the roomdepicted in FIG. 9A). In some embodiments, various IoT devices liketemperature sensors and motion sensors function via a hub orintermediary device, which may be controlled via a user's mobile phone.However, because these devices are all around users at all times, theremay be certain devices that a user may not want to provide access to.For example, a person may not want to provide access a smart camerainstalled in their bedroom under any circumstances. Or, for example,they may not want to provide access to the smart camera installed intheir bedroom unless the emergency is a fire.

Provided herein are systems and methods for managing access to connecteddevices during emergencies. In some embodiments, an emergency managementsystem (EMS) provides an emergency access management portal for users toregister their connected devices and assign various levels of permissionor access to those registered devices during emergencies. FIGS. 12A-12Ddepict various embodiments of an emergency access management portal1280. In some embodiments, the emergency access management portal 1280is a web application accessible through a standard internet browserusing a URL. In some embodiments, as depicted by FIG. 12A, the emergencyaccess management portal 1280 includes two tabs 1282, a Devices tab1282A and an Access tab 1282B. In the Devices tab 1282A, users canregister their devices under their account, such as by selecting the AddDevice button 1284. In some embodiments, when a user selects the AddDevice button 1284 to register a new device under their account, theemergency access management portal 1280 prompts the user to enterinformation about the device, such as the device's IP address, the makeand model of the device, the type of data or multimedia that the deviceis able to produce or capture, a name of the device, or a type of thedevice. In some embodiments, the EMS additionally or alternativelyprovides mobile applications that can be downloaded onto connecteddevices and used to register the devices with the emergency accessmanagement portal 1280. Once a device has been registered with theemergency access management portal 1280, the user can assign an accesslevel to the device, as described below. In the example depicted by FIG.12A, four devices 1285 have been registered under “Zakia's Devices”1281—“Bedroom Cam” 1285A, “Living Room Cam” 1285B, “Nest Thermo” 1285C,and “Apple Watch” 1285D. The name of the device, type of the device, andthe access level assigned to the device is displayed for each device1285. For example, “Bedroom Cam” 1285A is a Camera device type and hasbeen assigned an access level of Level 1. “Living Room Cam” 1285B isalso a Camera device type but has been assigned an access level of Level0. In some embodiments, a user can access or edit information or theaccess level of a device by selecting the device 1285 within theinterface (e.g., a graphical user interface) of the emergency accessmanagement portal 1280.

In some embodiments, the access level for a particular device can becustomized so that the emergency management system (EMS) can only accessthe particular device in certain situations, such as during particulartimes or types of emergencies. In some embodiments, access levels arecreated on a per device basis. For example, in some embodiments, when auser registers a device with the emergency access management portal1280, the user must select the particular circumstances for which theEMS can access the device during emergencies. In some embodiments,access levels are created independently of devices and can be laterassigned to particular devices, as depicted by FIG. 12A, wherein eachdevice 1285 has been assigned an access level. In some embodiments, auser can access an Access tab 1282B within the interface of theemergency access management portal 1280 to create, edit, or deleteaccess levels 1286 that can be assigned to the user's connected devices,as illustrated by FIG. 12B. In the example illustrated by FIG. 12B, auser has previously created two different access levels 1286, Level 01286A and Level 1 1286B. As shown in FIG. 12B, Level 0 1286A allows theEMS to access a device assigned to Level 0 during any and allemergencies, while Level 1 1286B allows the EMS to access a deviceassigned to Level 1 when the device is determined to be the mostrelevant device (as described above) or when the emergency type is afire. For example, as depicted in FIG. 12A, the device named “BedroomCam” 1285A has been assigned to Level 1. This might make sense for theuser because the bedroom is often a private space, but the user may wantto provide the EMS access to a camera in their bedroom during anemergency when the camera is the most relevant sensor (as describedabove) or when a visual of the emergency may be particularly helpful,such as in the case of a fire. In this way, access levels 1286 cancreate rules for when the EMS can access a particular device or sensor.For example, the rule defined by Level 1 1286B is “the EMS can access adevice assigned to Level 1 during an emergency when the device isdetermined to be the most relevant sensor OR when the emergency type isa fire.”

In some embodiments, a user can edit an access level 1286 by selectingthe access level 1286 within the emergency access management portal1280, as illustrated by FIG. 12C. In the example illustrated by FIG.12C, a user has selected to edit Level 1 1286B. In some embodiments, auser can edit the name of an access level 1288A and the permissions ofan access level 1288B. In the example illustrated by FIG. 12C, theemergency access management portal 1280 displays various permissions1288B that may be selected for an individual access level 1286,including (but not limited to): All Emergencies (e.g., the EMS canaccess a device assigned to this level during any and all emergencies),Fire (e.g., the EMS can access a device assigned to this level duringfire emergencies), Medical (e.g., the EMS can access a device assignedto this level during medical emergencies), By Request (e.g., the EMSmust request access to a device assigned to this level duringemergencies, as described below), Most Relevant (e.g., the EMS canaccess a device assigned to this level during an emergency when thedevice is determined to be the most relevant sensor, as describedabove), Proximity (e.g., the EMS can access a device assigned to thislevel during an emergency when the device is determined to be within athreshold distance from a location associated with the emergency), andNever (e.g., the EMS can never access a device assigned to this levelduring any emergency). In some embodiments, a user can select apermission as optional or required. For example, in the edited versionof Level 1 1286B illustrated by FIG. 12C, the Fire and Most Relevantpermissions have been selected as optional (marked with a check mark)and the Proximity permission has been selected as required. The newruled defined by the edited Level 1 would be “the EMS can access adevice assigned to Level 1 during an emergency when the device isdetermined to be within a threshold distance from a location associatedwith the emergency AND (when the device is determined to be the mostrelevant sensor OR when the emergency type is fire).” This means thatEMS could access a device assigned to the edited Level 1 during anemergency when the device is determined to be the most relevant sensoror when the emergency type is fire, but the device must always be withina threshold distance from a location associated with the emergency. Insome embodiments, the threshold distance is determined by the EMS (e.g.,based on the type of emergency, as described above). In someembodiments, the threshold distance is defined by the user (e.g., setwithin the emergency access management portal 1280). In someembodiments, a user can create new permissions 1288B that can beselected for an individual access level 1286, such as a Morningpermission (e.g., the EMS can access a device assigned to this levelduring AM hours). In some embodiments, after editing an access level1286, a user can save the edited access level by selecting the Savebutton 1288C. In some embodiments, a user can create a new access level1286 by selecting a create new access level button 1287. For example, asillustrated by FIG. 12D, a user has selected the create new access levelbutton 1287 and created a new access level, Level 2 1286C. The onlypermission selected for Level 2 1286C is By Request. The rule defined byLevel 2 1286C would be “the EMS must request to access a device assignedto this level during emergencies (as described below).”

In some embodiments, before the emergency management system (EMS) accessa device during an emergency to obtain emergency data or multimedia fromthe device and transmit the emergency data or multimedia to an emergencyservice provider (ESP) (as described above), the EMS must first identifyan access level assigned to the device and determine if the EMS haspermission to access the device for the emergency based on the accesslevel. For example, in the emergency illustrated by FIG. 7A, there arethree connected devices 710 in a living room during a fire 752: a smartcamera 710A, a smart thermometer 710B, and a smart speaker 710C. In thisexample, the owner of the three devices has registered all three deviceswith the emergency access management portal 1280 and assigned all threedevices a respective access level 1286. In this example, smart camera710A has been assigned Level 1 1286B; smart thermometer 710B has beenassigned Level 0 1286A, and smart speaker 710C has been assigned Level 21286C, as illustrated by FIG. 12D. In this example, after detecting thefire 752, the EMS identifies all three connected devices 710 as relevantsensors to the emergency (as described above) and attempts to obtainemergency data or multimedia from each of the three connected devices710 and transmit the emergency data or multimedia to an appropriate ESP.However, in this embodiment, the EMS must first identify the accesslevels 1286 assigned to the respective devices 710 and determine if theEMS has permission to access each of the devices 710.

In some embodiments, the EMS determines if the EMS has permission toaccess a device during an emergency according to the rule defined by anaccess level assigned to the device. In this example, the EMS identifiesthat the smart camera 710A has been assigned Level 1 1286B. The ruledefined by Level 1 1286B is “the EMS can access a device assigned toLevel 1 during an emergency when the device is determined to be the mostrelevant sensor OR when the emergency type is a fire” (as describedabove). Thus, in this example, the EMS does have permission to accessand obtain emergency data or multimedia from the smart camera 710Abecause the emergency type has been identified as a fire. Because theemergency type has been identified as a fire, whether or not the smartcamera 710A has been determined to be the most relevant sensor isrendered immaterial to determining if the EMS can access the smartcamera 710A. In this example, the EMS also identifies that the smartthermometer 710B has been assigned Level 0 1286A. The rule defined byLevel 0 1286 is “the EMS can access a device assigned to Level 1 duringany and all emergencies.” Thus, in this example, the EMS does havepermission to access and obtain emergency data or multimedia from thesmart thermometer 710B. Finally, in this example, the EMS alsoidentifies that the smart speaker 710C has been assigned Level 2 1286C.The rule defined by Level 2 1286C is “the EMS must request access to adevice assigned to Level 2 during emergencies.” Thus, in this example,the EMS does not have permission to access and obtain multimedia fromthe smart speaker 710C, but may request access, as described below. Forexample, the EMS can deliver a multimedia inquiry (e.g., “911 would liketo access this speaker, please confirm”) to the smart speaker to beplayed audibly through the smart speaker. A person (e.g., the man orwoman within the family 758C illustrated by FIG. 7A) may confirm themultimedia inquiry by audibly responding in the affirmative (e.g.,“access granted”), thereby giving permission to the EMS to access andobtain emergency data or multimedia from the smart speaker 710C. Or, forexample, the EMS can deliver a multimedia inquiry to a second deviceassociated with the smart speaker (e.g., a cell phone or smart watchbelonging to the man or woman within the family 758C) to request accessto the smart speaker 710C, as described below.

Requesting Emergency Data and Multimedia

In some embodiments, as mentioned above, when the EMS identifies adevice or sensor associated with an emergency, the EMS must requestaccess to the device or sensor before accessing and obtaining emergencydata or multimedia from the device or sensor. In some embodiments, amethod for transmitting multimedia from an electronic device to anemergency service provider (ESP) by an emergency management system (EMS)comprises: (a) identifying an electronic device associated with anemergency alert; (b) delivering a multimedia inquiry to the electronicdevice; (c) receiving a confirmation of the multimedia inquiry from theelectronic device; (d) determining an emergency service provider (ESP)to receive multimedia from the electronic device using a location of theelectronic device and a geofence system; (e) establishing acommunication link between the electronic device and the ESP; and (f)transmitting multimedia content from the electronic device to the ESPvia the communication link. In some embodiments, the multimedia contentcomprises a video feed. In some embodiments, transmitting the multimediacontent from the electronic device to the ESP via the communication linkcomprises: (a) hosting the video feed at a remote server; and (b)providing a web link to the ESP, wherein the web link points to thevideo feed hosted on the remote server. In some embodiments, identifyingthe electronic device associated with the emergency alert comprisesautonomously receiving the emergency alert from the electronic devicewhen the electronic device executes an emergency phone call. In someembodiments, the method further comprises: (a) displaying a multimediaaccess button within an emergency response application executed on acomputing device at the ESP; (b) establishing the communication linkbetween the electronic device and the ESP in response to receivingselection of the multimedia access button; and (c) displaying themultimedia content through the emergency response application. In someembodiments, the method further comprises: (a) displaying a media accessbutton within an emergency response application executed on a computingdevice at the ESP; (b) receiving selection of the multimedia accessbutton; and (c) delivering the multimedia inquiry to the electronicdevice in response to receiving the selection of the multimedia accessbutton. In some embodiments, identifying the electronic deviceassociated with the emergency alert comprises receiving an emergencydata request from the ESP, wherein the emergency data request isgenerated and transmitted by the emergency response application inresponse to receiving the selection of the multimedia access button andwherein the emergency data request comprises an identifier of theelectronic device. In some embodiments, the identifier of the electronicdevice is a phone number. In some embodiments, the emergency responseapplication is a web application accessible via a web browser using aURL. In some embodiments, the emergency alert is an emergency call madefrom the electronic device and wherein identifying the electronic deviceassociated with the emergency alert comprises [autonomously] detectingthe emergency call made by the electronic device. In some embodiments,determining the ESP to receive multimedia from the electronic devicecomprises: (a) retrieving a plurality of geofences associated with aplurality of ESPs including the ESP; and (b) determining that thelocation of the electronic device is within a geofence associated withthe ESP. In some embodiments, the multimedia inquiry is an SMS messagecomprising a web link and wherein confirmation of the multimedia inquirycomprises selection of the web link. In some embodiments, the multimediainquiry is a push notification and wherein confirmation of themultimedia inquiry comprises selection of the push notification. In someembodiments, the method further comprises: (a) sampling a connectionquality of the transmission of the multimedia content; and (b) inresponse to the connection quality falling below a threshold value,downgrading the multimedia content. In some embodiments, the multimediamedia content comprises a video feed; and the video feed is downgradedto image stills or reduced framerate or resolution. In some embodiments,the multimedia content comprises a video file; and the video file isdowngraded to a summarization. In some embodiments, the multimediacontent comprises an audio feed or audio file; and the audio feed oraudio file is downgraded to a text transcription.

For example, FIG. 13 illustrates a multimedia inquiry delivered to anelectronic device. In some embodiments, the emergency management system(EMS) identifies an electronic device (e.g., a cell phone) as associatedwith an emergency alert when an emergency call is made from theelectronic device, which generates and transmits the emergency alert tothe EMS in parallel. Thus, the EMS identifies the electronic device asassociated with an emergency alert by receiving the emergency alert fromthe electronic device. In some embodiments, the electronic devicegenerates the emergency alert and transmits the emergency alert to athird-party server system, which then transmits the emergency alert tothe EMS, allowing the EMS to identify the electronic device asassociated with the emergency alert. However, in some embodiments, theEMS identifies an electronic device as associated with an emergencyalert by receiving an emergency data request or multimedia requestincluding an identifier of the electronic device (e.g., a phone number)from an emergency service provider (ESP), as described below. In theexample illustrated by FIG. 13, an emergency call has been made fromelectronic device 1310, thereby generating an emergency alert. The EMShas identified the electronic device as associated with the emergencyalert, as described above. After identifying the electronic device asassociated with the emergency alert, the EMS has delivered a multimediainquiry 1304 to the electronic device 1310 to request access to theelectronic device 1310 from a user of the electronic device. In thisembodiment, the multimedia inquiry 1304 is a text message that includesa written request for multimedia (“This is 9-1-1 in New York, N.Y.Please share live video using this link:”) and a weblink (e.g., a URL)that the user can use to confirm the multimedia inquiry 1304 and allowthe EMS to access the electronic device 1310. In some embodiments, theEMS autonomously delivers a multimedia inquiry 1304 to an electronicdevice in response to receiving an emergency alert associated with theelectronic device. In some embodiments, the EMS delivers a multimediainquiry 1304 to an electronic device only in response to receiving amultimedia request from an ESP. In this example, the electronic device1310 displays a notification 1304A regarding the multimedia inquiry 1304during the emergency call that the user can select to view themultimedia inquiry text message 1304B within the user's message viewer.The user can then select the link within the message viewer to confirmthe multimedia inquiry 1304 and allow the EMS to access the electronicdevice 1310. In some embodiments, the multimedia inquiry 1304 is sent toan electronic device as a push notification or an audible message, asdescribed above. However, the multimedia inquiry 1304 may take anysuitable or necessary form. In some embodiments, the EMS is integratedinto the electronic device, and the multimedia inquiry 1304 is presentedto the user in response to the user making an emergency call from theelectronic device.

FIG. 14 illustrates a web application provided by an emergencymanagement system (EMS) for obtaining multimedia content from anelectronic device. As illustrated in FIG. 14, in some embodiments, whenthe user selects the link within their message viewer, the electronicdevice opens an internet browser and navigates to the address of theweblink, where the EMS provides a web application 1480 for sharingmultimedia content from the electronic device to an emergency serviceprovider (ESP). In this example, the EMS is attempting to obtain videofrom a camera integrated into the electronic device 1410 and transmitthe video to an ESP. In some embodiments, the web application 1480prompts the user to provide access to the camera integrated into theelectronic device, as illustrated by FIG. 14. In some embodiments, theweb application 1480 includes a Share Video button 1406A that the usercan select to begin recording video that will be sent to an ESP, asillustrated by FIG. 14. In this example, the electronic device records avideo of a building fire 1452. In some embodiments, the web application1480 includes an End button 1406B that the user can select to stopsharing video. In some embodiments, in which the EMS is integrated intothe electronic device, when the user confirms the multimedia inquiry, anative application is executed on the electronic device that performsany and all of the functions of the web application accessed using theweblink.

In some embodiments, when the EMS attempts to access a first electronicdevice associated with an emergency alert, the EMS can deliver amultimedia inquiry to a second electronic device associated with theemergency alert or the first electronic device. For example, FIG. 15illustrates a fire 1552 that has ignited in a living room of a homewhile all of the occupants were out of the home. In this example, anemergency alert was generated by the smart thermostat 1510B after thesmart thermostat 1510B autonomously detected the fire 1552. In thisexample, the EMS then identifies both smart camera 1510A and smartspeaker 1510C as relevant sensors (as described above). The EMS attemptsto access the smart camera 1510A but the smart camera 1510A is assignedan access level that requires the EMS to request access to the smartcamera 1510A (as described above). In this example, the EMS identifiescell phone 1510D as associated with the smart camera 1510A (e.g., bothare registered to the same account), and delivers a multimedia inquiry1504 to the cell phone 1510D to request access to the smart camera1510A. In this example, the multimedia inquiry 1504 includes a link thatthe user of the cell phone 1510D can select to authorize the EMS toaccess the smart camera 1510A.

FIGS. 16A and 16B illustrate examples of a graphical user interface(GUI) of an emergency response application. In some embodiments, theemergency response application 1680 is provided by the emergencymanagement system (EMS). In some embodiments, the emergency responseapplication 1680 is a web application accessible through a standardinternet browser using a URL. In some embodiments, the EMS provides anemergency response application 1680 to share and display emergency dataand multimedia to emergency service providers (ESP; e.g., a publicsafety answering point (PSAP)). In some embodiments, when an ESP (e.g.,a PSAP) receives an emergency alert (e.g., an emergency call), a memberof the ESP (e.g., a PSAP call taker) can use the emergency responseapplication 1680 to receive emergency data and multimedia associatedwith the emergency alert from the EMS. For example, in some embodiments,when a PSAP receives an emergency call, a call taker at the PSAP canenter the phone number into the emergency response application 1680 inthe Device Identifier input box 1697A and select the search button 1697Bto transmit an emergency data request including a device identifierassociated with the emergency alert (e.g., the phone number of the phonethat the emergency call was made from) to the EMS, as described above.In some embodiments, when the EMS receives an emergency data requestincluding a device identifier from an ESP, the EMS returns any and allemergency data associated with the device identifier to the ESP, such asany available location associated with the device identifier, asdescribed above. In the example illustrated in FIG. 16A, the EMS hasreturned a location associated with an electronic device associated withthe emergency alert to an ESP. The location is displayed graphicallywithin a map 1698 as an emergency location marker 1699A. In thisexample, the location is also displayed textually at Current Location1699B. In some embodiments, when the EMS receives an emergency alertincluding a location associated with the emergency alert, the EMSdetermines an appropriate ESP to receive emergency data or multimediaassociated with the emergency alert using a geofence system, asdescribed above, and autonomously pushes emergency data or multimediaassociated with the emergency alert to the ESP through the emergencyresponse application 1680 without first receiving an emergency datarequest or multimedia request. In some embodiments, the ESP uses ageospatial query (e.g., the emergency location 1699A) to locatemultimedia sources in the vicinity (e.g., the smart camera representedby multimedia location 1699C). In some embodiments, a multimedia deviceis in the vicinity of the emergency location when it is within a certaindistance (e.g., threshold distance or vicinity) as described elsewherein the specification herein. In some embodiments, a multimedia device isin the vicinity of the emergency location when it is within about 1, 5,10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 150, 200, 250, 300, 350, 400,450, 500, 600, 700, 800, 900, or 1000 meters from the emergencylocation. In some embodiments, the user or owner of the triggeringdevice (the device that has triggered the emergency) sets the distanceparameter for searching for multimedia devices in the vicinity. In someembodiments, the user or owner of the triggering device identifiesspecific multimedia in the vicinity for obtaining contextual information(e.g., multimedia data) during an emergency.

In some embodiments, the emergency response application 1680 includes amultimedia access button 1691. In some embodiments, the multimediaaccess button 1691 is only displayed within the emergency responseapplication 1680 after the EMS has delivered a multimedia inquiry to anelectronic device associated with the emergency alert for whichemergency data is being viewed or requested and the multimedia inquiryhas been confirmed. In such an embodiment, a user of the emergencyresponse application 1680 can select the multimedia access button 1691to begin receiving multimedia from an electronic device associated withthe emergency alert for which emergency data is being viewed orrequested. In some embodiments, the EMS displays the multimedia accessbutton 1691 before delivering a multimedia inquiry to an electronicdevice associated with an emergency alert, and a user of the emergencyresponse application 1680 can select the multimedia access button 1691to transmit a multimedia request including an identifier of anelectronic device associated with the emergency alert (e.g., a phonenumber of a phone from which an emergency call was made). In such anembodiment, the EMS can then deliver a multimedia inquiry to theelectronic device, obtain multimedia content from the electronic device,and transmit multimedia content from the electronic device to therequesting ESP. FIG. 16B illustrates an example of multimedia contentbeing transmitted to an ESP through the emergency response application1680. In some embodiments, the multimedia content is displayed in amultimedia section 1692 of the emergency response application 1680. Insome embodiments, when the multimedia content includes a video, thevideo may be enlarged by selecting an enlarge button 1695. In theexample illustrated by FIG. 16B, the multimedia content transmittedthrough the emergency response application 1680 is the video of thebuilding fire recorded by the electronic device illustrated in FIG. 14.In some embodiments, video transmitted to an ESP is presented in awindow separate from the emergency response application 1680. In someembodiments, video transmitted to an ESP is hosted on a separate server.In some embodiments, a link to the separate server is provided to theESP through the emergency response application 1680. In someembodiments, when the device from which multimedia is available orrequested is not the device that generated the emergency alert, the map1698 displays a multimedia location marker 1699C representing thelocation of the device from which multimedia is available or requested.In some embodiments, the multimedia location marker 1699C is visuallydistinct from the emergency location marker 1699A. For example, themultimedia location marker 1699C may be displayed in a different coloror size than the emergency location marker 1699A.

The present methods and systems disclose partially or fully integratedsolutions emergency management systems, dispatch centers and electronicdevices. Previous systems were not integrated and could not providesecured pathways, user privacy and software compatibilities. In someembodiments, partial integration with a dispatch center or firstresponder comprises adding the emergency management system as a “trustedlink” on their systems or devices. In some embodiments, end-to-endsoftware solutions are beneficial, e.g., for video feed from a sensor tobe available to first responders.

Algorithms

In some embodiments, the systems, methods, and media described hereinuse one or more algorithms analyzing multimedia content. In someembodiments, machine learning algorithms are used for trainingprediction models and/or making predictions such as predicting anemergency or relevance to an emergency. Various algorithms can be usedto generate models that are used to identify a first set of sensorspertinent to an emergency and/or a second set of sensors relevant to theemergency. In some instances, machine learning methods are applied tothe generation of such models.

In some embodiments, a machine learning algorithm uses a supervisedlearning approach. In supervised learning, the algorithm generates afunction from labeled training data. Each training example is a pairconsisting of an input object and a desired output value. In someembodiments, an optimal scenario allows for the algorithm to correctlydetermine the class labels for unseen instances. In some embodiments, asupervised learning algorithm requires the user to determine one or morecontrol parameters. These parameters are optionally adjusted byoptimizing performance on a subset, called a validation set, of thetraining set. After parameter adjustment and learning, the performanceof the resulting function is optionally measured on a test set that isseparate from the training set. Regression methods are commonly used insupervised learning. Accordingly, supervised learning allows for a modelor classifier to be generated or trained with training data in which theexpected output is known in advance such as when the relevance is known(e.g., based on historical relevance scores).

In some embodiments, a machine learning algorithm uses an unsupervisedlearning approach. In unsupervised learning, the algorithm generates afunction to describe hidden structures from unlabeled data (e.g., aclassification or categorization is not included in the observations).Since the examples given to the learner are unlabeled, there is noevaluation of the accuracy of the structure that is output by therelevant algorithm. Approaches to unsupervised learning include:clustering, anomaly detection, and neural networks.

In some embodiments, a machine learning algorithm learns in batchesbased on the training dataset and other inputs for that batch. In otherembodiments, the machine learning algorithm performs on-line learningwhere the weights and error calculations are constantly updated.

In some embodiments, a machine learning algorithm is applied to new orupdated emergency data to be re-trained to generate a new predictionmodel. In some embodiments, a machine learning algorithm or model isre-trained periodically. In some embodiments, a machine learningalgorithm or model is re-trained non-periodically. In some embodiments,a machine learning algorithm or model is re-trained at least once a day,a week, a month, or a year or more. In some embodiments, a machinelearning algorithm or model is re-trained at least once every 1, 2, 3,4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22,23, 24, 25, 26, 27, 28, 29, or 30 days or more.

In some instances, machine learning methods are applied to select, froma plurality of models generated, one or more particular models that aremore applicable to certain attributes. In some embodiments, differentmodels are generated depending on the distinct sets of attributesobtained for various communications.

In some embodiments, the classifier or trained algorithm of the presentdisclosure comprises one feature space. In some cases, the classifiercomprises two or more feature spaces. In some embodiments, the two ormore feature spaces are distinct from one another. In variousembodiments, each feature space comprise types of attributes associatedwith a communication such as network carrier, make/model of thecommunication device, reported geographic location, or other informationrelevant to location accuracy. In some embodiments, the accuracy of theclassification or prediction is improved by combining two or morefeature spaces in a classifier instead of using a single feature space.The attributes generally make up the input features of the feature spaceand are labeled to indicate the classification of each communication forthe given set of input features corresponding to that communication. Inmany cases, the classification is the location accuracy metric or aprediction of location accuracy.

In some embodiments, an algorithm utilizes a predictive model such as aneural network, a decision tree, a support vector machine, or otherapplicable model. Using the training data, an algorithm is able to forma classifier for generating a classification or prediction according torelevant features. The features selected for classification can beclassified using a variety of viable methods. In some embodiments, thetrained algorithm comprises a machine learning algorithm. In someembodiments, the machine learning algorithm is selected from at leastone of a supervised, semi-supervised and unsupervised learning, such as,for example, a support vector machine (SVM), a Naïve Bayesclassification, a random forest, an artificial neural network, adecision tree, a K-means, learning vector quantization (LVQ), regressionalgorithm (e.g., linear, logistic, multivariate), association rulelearning, deep learning, dimensionality reduction and ensemble selectionalgorithms. In some embodiments, the machine learning algorithm is asupport vector machine (SVM), a Naïve Bayes classification, a randomforest, or an artificial neural network. Machine learning techniquesinclude bagging procedures, boosting procedures, random forestalgorithms, and combinations thereof.

In some embodiments, a machine learning algorithm such as a classifieris tested using data that was not used for training to evaluate itspredictive ability. In some embodiments, the predictive ability of theclassifier is evaluated using one or more metrics. These metrics includeaccuracy, specificity, sensitivity, positive predictive value, negativepredictive value, which are determined for a classifier by testing itagainst a set of independent cases (e.g., communications). In someinstances, an algorithm has an accuracy of at least about 75%, 80%, 85%,90%, 95% or more, including increments therein, for at least about 50,60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, or 200independent cases, including increments therein. In some instances, analgorithm has a specificity of at least about 75%, 80%, 85%, 90%, 95% ormore, including increments therein, for at least about 50, 60, 70, 80,90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, or 200 independentcases, including increments therein. In some instances, an algorithm hasa sensitivity of at least about 75%, 80%, 85%, 90%, 95% or more,including increments therein, for at least about 50, 60, 70, 80, 90,100, 110, 120, 130, 140, 150, 160, 170, 180, 190, or 200 independentcases, including increments therein. In some instances, an algorithm hasa positive predictive value of at least about 75%, 80%, 85%, 90%, 95% ormore, including increments therein, for at least about 50, 60, 70, 80,90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, or 200 independentcases, including increments therein. In some instances an algorithm hasa negative predictive value of at least about 75%, 80%, 85%, 90%, 95% ormore, including increments therein, for at least about 50, 60, 70, 80,90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, or 200 independentcases, including increments therein.

In some embodiments, the multimedia content undergoes natural languageprocessing using one or more machine learning algorithms. In someembodiments, the one or more machine learning algorithms utilize wordembedding(s) that map words or phrases to vectors of real numbers. Insome embodiments, the mapping is generated by a neural network. In someembodiments, a machine learning algorithm is applied to parse the textobtained from multimedia content (e.g., extracted text from a video oraudio recording). In some embodiments, a machine learning algorithm isapplied to segment words into morphemes and identify the class of themorphemes. In some embodiments, a machine learning algorithm is appliedto identify and/or tag the part of speech for the words in themultimedia content (e.g., tagging a word as a noun, verb, adjective, oradverb). In some embodiments, a machine learning algorithm is applied toclassify multimedia content into a category such as relevance (e.g.,relevant or irrelevant to an emergency). In some embodiments, theapplication applies at least one machine learning algorithm tomultimedia content to determine an emergency type (e.g., injury oraccident, medical problem, shooting, violent crime, robbery, tornado, orfire) and/or emergency level (e.g., safe, low, medium, high).

Digital Processing Device

In some embodiments, the platforms, media, methods and applicationsdescribed herein include a digital processing device, a processor, oruse of the same. In further embodiments, the digital processing deviceincludes one or more hardware central processing units (CPU) that carryout the device's functions. In still further embodiments, the digitalprocessing device further comprises an operating system configured toperform executable instructions. In some embodiments, the digitalprocessing device is optionally connected a computer network. In furtherembodiments, the digital processing device is optionally connected tothe Internet such that it accesses the World Wide Web. In still furtherembodiments, the digital processing device is optionally connected to acloud computing infrastructure. In other embodiments, the digitalprocessing device is optionally connected to an intranet. In otherembodiments, the digital processing device is optionally connected to adata storage device.

In accordance with the description herein, suitable digital processingdevices include, by way of non-limiting examples, server computers,desktop computers, laptop computers, notebook computers, sub-notebookcomputers, netbook computers, netpad computers, set-top computers,handheld computers, Internet appliances, mobile smartphones, tabletcomputers, personal digital assistants, video game consoles, andvehicles. Those of skill in the art will recognize that many smartphonesare suitable for use in the system described herein. Those of skill inthe art will also recognize that select televisions, video players, anddigital music players with optional computer network connectivity aresuitable for use in the system described herein. Suitable tabletcomputers include those with booklet, slate, and convertibleconfigurations, known to those of skill in the art.

In some embodiments, the digital processing device includes an operatingsystem configured to perform executable instructions. The operatingsystem is, for example, software, including programs and data, whichmanages the device's hardware and provides services for execution ofapplications. Those of skill in the art will recognize that suitableserver operating systems include, by way of non-limiting examples,FreeBSD, OpenBSD, NetBSD®, Linux, Apple® Mac OS X Server®, Oracle®Solaris®, Windows Server®, and Novell® NetWare®. Those of skill in theart will recognize that suitable personal computer operating systemsinclude, by way of non-limiting examples, Microsoft® Windows®, Apple®Mac OS X®, UNIX®, and UNIX-like operating systems such as GNU/Linux®. Insome embodiments, the operating system is provided by cloud computing.Those of skill in the art will also recognize that suitable mobile smartphone operating systems include, by way of non-limiting examples, Nokia®Symbian® OS, Apple® iOS®, Research In Motion® BlackBerry OS®, Google®Android®, Microsoft® Windows Phone® OS, Microsoft® Windows Mobile® OS,Linux®, and Palm® WebOS®.

In some embodiments, the device includes a storage and/or memory device.The storage and/or memory device is one or more physical apparatusesused to store data or programs on a temporary or permanent basis. Insome embodiments, the device is volatile memory and requires power tomaintain stored information. In some embodiments, the device isnon-volatile memory and retains stored information when the digitalprocessing device is not powered. In further embodiments, thenon-volatile memory comprises flash memory. In some embodiments, thenon-volatile memory comprises dynamic random-access memory (DRAM). Insome embodiments, the non-volatile memory comprises ferroelectric randomaccess memory (FRAM). In some embodiments, the non-volatile memorycomprises phase-change random access memory (PRAM). In some embodiments,the non-volatile memory comprises magnetoresistive random-access memory(MRAM). In other embodiments, the device is a storage device including,by way of non-limiting examples, CD-ROMs, DVDs, flash memory devices,magnetic disk drives, magnetic tapes drives, optical disk drives, andcloud computing based storage. In further embodiments, the storageand/or memory device is a combination of devices such as those disclosedherein.

In some embodiments, the digital processing device includes a display tosend visual information to a subject. In some embodiments, the displayis a cathode ray tube (CRT). In some embodiments, the display is aliquid crystal display (LCD). In further embodiments, the display is athin film transistor liquid crystal display (TFT-LCD). In someembodiments, the display is an organic light emitting diode (OLED)display. In various further embodiments, on OLED display is apassive-matrix OLED (PMOLED) or active-matrix OLED (AMOLED) display. Insome embodiments, the display is a plasma display. In some embodiments,the display is E-paper or E ink. In other embodiments, the display is avideo projector. In still further embodiments, the display is acombination of devices such as those disclosed herein.

In some embodiments, the digital processing device includes an inputdevice to receive information from a subject. In some embodiments, theinput device is a keyboard. In some embodiments, the input device is apointing device including, by way of non-limiting examples, a mouse,trackball, track pad, joystick, game controller, or stylus. In someembodiments, the input device is a touch screen or a multi-touch screen.In other embodiments, the input device is a microphone to capture voiceor other sound input. In other embodiments, the input device is a videocamera or other sensor to capture motion or visual input. In furtherembodiments, the input device is a Kinect, Leap Motion, or the like. Instill further embodiments, the input device is a combination of devicessuch as those disclosed herein.

Non-Transitory Computer-Readable Storage Medium

In some embodiments, the platforms, media, methods and applicationsdescribed herein include one or more non-transitory computer readablestorage media encoded with a program including instructions executableby the operating system of an optionally networked digital processingdevice. In further embodiments, a computer readable storage medium is atangible component of a digital processing device. In still furtherembodiments, a computer readable storage medium is optionally removablefrom a digital processing device. In some embodiments, a computerreadable storage medium includes, by way of non-limiting examples,CD-ROMs, DVDs, flash memory devices, solid state memory, magnetic diskdrives, magnetic tape drives, optical disk drives, cloud computingsystems and services, and the like. In some cases, the program andinstructions are permanently, substantially permanently,semi-permanently, or non-transitorily encoded on the media.

Computer Program

In some embodiments, the platforms, media, methods and applicationsdescribed herein include at least one computer program, or use of thesame. A computer program includes a sequence of instructions, executablein the digital processing device's CPU, written to perform a specifiedtask. Computer readable instructions may be implemented as programmodules, such as functions, objects, Application Programming Interfaces(APIs), data structures, and the like, that perform particular tasks orimplement particular abstract data types. In light of the disclosureprovided herein, those of skill in the art will recognize that acomputer program may be written in various versions of variouslanguages.

The functionality of the computer readable instructions may be combinedor distributed as desired in various environments. In some embodiments,a computer program comprises one sequence of instructions. In someembodiments, a computer program comprises a plurality of sequences ofinstructions. In some embodiments, a computer program is provided fromone location. In other embodiments, a computer program is provided froma plurality of locations. In various embodiments, a computer programincludes one or more software modules. In various embodiments, acomputer program includes, in part or in whole, one or more webapplications, one or more mobile applications, one or more standaloneapplications, one or more web browser plug-ins, extensions, add-ins, oradd-ons, or combinations thereof.

Web Application

In some embodiments, a computer program includes a web application. Inlight of the disclosure provided herein, those of skill in the art willrecognize that a web application, in various embodiments, utilizes oneor more software frameworks and one or more database systems. In someembodiments, a web application is created upon a software framework suchas Microsoft® .NET or Ruby on Rails (RoR). In some embodiments, a webapplication utilizes one or more database systems including, by way ofnon-limiting examples, relational, non-relational, object oriented,associative, and XML database systems. In further embodiments, suitablerelational database systems include, by way of non-limiting examples,Microsoft® SQL Server, mySQL™, and Oracle®. Those of skill in the artwill also recognize that a web application, in various embodiments, iswritten in one or more versions of one or more languages. A webapplication may be written in one or more markup languages, presentationdefinition languages, client-side scripting languages, server-sidecoding languages, database query languages, or combinations thereof. Insome embodiments, a web application is written to some extent in amarkup language such as Hypertext Markup Language (HTML), ExtensibleHypertext Markup Language (XHTML), or eXtensible Markup Language (XML).In some embodiments, a web application is written to some extent in apresentation definition language such as Cascading Style Sheets (CSS).In some embodiments, a web application is written to some extent in aclient-side scripting language such as Asynchronous Javascript and XML(AJAX), Flash® Actionscript, Javascript, or Silverlight®. In someembodiments, a web application is written to some extent in aserver-side coding language such as Active Server Pages (ASP),ColdFusion®, Perl, Java™, JavaServer Pages (JSP), Hypertext Preprocessor(PHP), Python™, Ruby, Tcl, Smalltalk, WebDNA®, or Groovy. In someembodiments, a web application is written to some extent in a databasequery language such as Structured Query Language (SQL). In someembodiments, a web application integrates enterprise server productssuch as IBM® Lotus Domino®. In some embodiments, a web applicationincludes a media player element. In various further embodiments, a mediaplayer element utilizes one or more of many suitable multimediatechnologies including, by way of non-limiting examples, Adobe® Flash®,HTML 5, Apple® QuickTime®, Microsoft Silverlight®, Java™, and Unity

Mobile Application

In some embodiments, a computer program includes a mobile applicationprovided to a mobile digital processing device. In some embodiments, themobile application is provided to a mobile digital processing device atthe time it is manufactured. In other embodiments, the mobileapplication is provided to a mobile digital processing device via thecomputer network described herein.

In view of the disclosure provided herein, a mobile application iscreated by techniques known to those of skill in the art using hardware,languages, and development environments known to the art. Those of skillin the art will recognize that mobile applications are written inseveral languages. Suitable programming languages include, by way ofnon-limiting examples, C, C++, C#, Objective-C, Java™, Javascript,Pascal, Object Pascal, Python™, Ruby, VB.NET, WML, and XHTML/HTML withor without CSS, or combinations thereof.

Suitable mobile application development environments are available fromseveral sources. Commercially available development environmentsinclude, by way of non-limiting examples, AirplaySDK, alcheMo,Appcelerator®, Celsius, Bedrock, Flash Lite, .NET Compact Framework,Rhomobile, and WorkLight Mobile Platform. Other development environmentsare available without cost including, by way of non-limiting examples,Lazarus, MobiFlex, MoSync, and Phonegap. Also, mobile devicemanufacturers distribute software developer kits including, by way ofnon-limiting examples, iPhone and iPad (iOS) SDK, Android™ SDK,BlackBerry® SDK, BREW SDK, Palm® OS SDK, Symbian SDK, webOS SDK, andWindows® Mobile SDK.

Those of skill in the art will recognize that several commercial forumsare available for distribution of mobile applications including, by wayof non-limiting examples, Apple® App Store, Android™ Market, BlackBerry®App World, App Store for Palm devices, App Catalog for webOS, Windows®Marketplace for Mobile, Ovi Store for Nokia® devices, Samsung® Apps, andNintendo® DSi Shop.

Standalone Application

In some embodiments, a computer program includes a standaloneapplication, which is a program that is run as an independent computerprocess, not an add-on to an existing process, e.g., not a plug-in.Those of skill in the art will recognize that standalone applicationsare often compiled. A compiler is a computer program(s) that transformssource code written in a programming language into binary object codesuch as assembly language or machine code. Suitable compiled programminglanguages include, by way of non-limiting examples, C, C++, Objective-C,COBOL, Delphi, Eiffel, Java™, Lisp, Python™, Visual Basic, and VB .NET,or combinations thereof. Compilation is often performed, at least inpart, to create an executable program. In some embodiments, a computerprogram includes one or more executable complied applications.

Software Modules

In some embodiments, the platforms, media, methods and applicationsdescribed herein include software, server, and/or database modules, oruse of the same. In view of the disclosure provided herein, softwaremodules are created by techniques known to those of skill in the artusing machines, software, and languages known to the art. The softwaremodules disclosed herein are implemented in a multitude of ways. Invarious embodiments, a software module comprises a file, a section ofcode, a programming object, a programming structure, or combinationsthereof. In further various embodiments, a software module comprises aplurality of files, a plurality of sections of code, a plurality ofprogramming objects, a plurality of programming structures, orcombinations thereof. In various embodiments, the one or more softwaremodules comprise, by way of non-limiting examples, a web application, amobile application, and a standalone application. In some embodiments,software modules are in one computer program or application. In otherembodiments, software modules are in more than one computer program orapplication. In some embodiments, software modules are hosted on onemachine. In other embodiments, software modules are hosted on more thanone machine. In further embodiments, software modules are hosted oncloud computing platforms. In some embodiments, software modules arehosted on one or more machines in one location. In other embodiments,software modules are hosted on one or more machines in more than onelocation.

Databases

In some embodiments, the platforms, systems, media, and methodsdisclosed herein include one or more databases, or use of the same. Inview of the disclosure provided herein, those of skill in the art willrecognize that many databases are suitable for storage and retrieval ofbarcode, route, parcel, subject, or network information. In variousembodiments, suitable databases include, by way of non-limitingexamples, relational databases, non-relational databases, objectoriented databases, object databases, entity-relationship modeldatabases, associative databases, and XML, databases. In someembodiments, a database is internet-based. In further embodiments, adatabase is web-based. In still further embodiments, a database is cloudcomputing-based. In other embodiments, a database is based on one ormore local computer storage devices.

Web Browser Plug-in

In some embodiments, the computer program includes a web browserplug-in. In computing, a plug-in is one or more software components thatadd specific functionality to a larger software application. Makers ofsoftware applications support plug-ins to enable third-party developersto create abilities which extend an application, to support easilyadding new features, and to reduce the size of an application. Whensupported, plug-ins enable customizing the functionality of a softwareapplication. For example, plug-ins are commonly used in web browsers toplay video, generate interactivity, scan for viruses, and displayparticular file types. Those of skill in the art will be familiar withseveral web browser plug-ins including, Adobe® Flash® Player, Microsoft®Silverlight®, and Apple® QuickTime®. In some embodiments, the toolbarcomprises one or more web browser extensions, add-ins, or add-ons. Insome embodiments, the toolbar comprises one or more explorer bars, toolbands, or desk bands.

In view of the disclosure provided herein, those of skill in the artwill recognize that several plug-in frameworks are available that enabledevelopment of plug-ins in various programming languages, including, byway of non-limiting examples, C++, Delphi, Java™, PHP, Python™, and VB.NET, or combinations thereof.

Web browsers (also called Internet browsers) are software applications,designed for use with network-connected digital processing devices, forretrieving, presenting, and traversing information resources on theWorld Wide Web. Suitable web browsers include, by way of non-limitingexamples, Microsoft® Internet Explorer®, Mozilla® Firefox®, Google®Chrome, Apple® Safari®, Opera Software® Opera®, and KDE Konqueror. Insome embodiments, the web browser is a mobile web browser. Mobile webbrowsers (also called microbrowsers, mini-browsers, and wirelessbrowsers) are designed for use on mobile digital processing devicesincluding, by way of non-limiting examples, handheld computers, tabletcomputers, netbook computers, subnotebook computers, smartphones, musicplayers, personal digital assistants (PDAs), and handheld video gamesystems. Suitable mobile web browsers include, by way of non-limitingexamples, Google® Android® browser, RIM BlackBerry® Browser, Apple®Safari®, Palm® Blazer, Palm® WebOS® Browser, Mozilla® Firefox® formobile, Microsoft® Internet Explorer® Mobile, Amazon® Kindle® Basic Web,Nokia® Browser, Opera Software® Opera® Mobile, and Sony® PSP™ browser.

Certain Terminologies

Unless otherwise defined, all technical terms used herein have the samemeaning as commonly understood by one of ordinary skill in the art towhich this invention belongs. As used in this specification and theappended claims, the singular forms “a,” “an,” and “the” include pluralreferences unless the context clearly dictates otherwise. Any referenceto “or” herein is intended to encompass “and/or” unless otherwisestated.

As used herein, a “device” or “electronic device” is a digitalprocessing device designed with one or more functionalities such as, forexample, a communication device or sensor device. A “triggering device”refers to an electronic device with a communication component, whichwill allow it to send and receive information over a wireless channel, awired channel, or any combination thereof (e.g., sending/receivinginformation over the Internet). Examples of triggering devices include amobile phone (e.g., a smartphone), a laptop, a desktop, a tablet, aradio (e.g., a two-way radio), and a vehicular communication system. Insome embodiments, a triggering device includes a car security system(e.g., OnStar®), a home security system, or a home control system (e.g.,a networked control system for providing network controlled and/or smarttemperature control such as a Wi-Fi smart thermostat, lighting,entertainment, and/or door control, such as Nest®). In some embodiments,the triggering device is a physical panic button or software “panic”button. In some embodiments, a triggering device is an Internet ofThings (IoT) device. In some embodiments, an electronic device is asensor for sensing environmental or health indicators. In someembodiments, the sensor may include a sensing component and acommunication component. In some embodiments, the triggering device is asensor in a sensor network or a device that controls a sensor network.

In some embodiments, a triggering device is a wearable device (e.g., acommunication device worn by a user, such as an Apple Watch). In someembodiments, a triggering device (e.g., a wearable device) comprises oneor more sensors. The one or more sensors may include, but are notlimited to: a gyroscope, an accelerometer, a thermometer, a heart ratesensor, a barometer, or a hematology analyzer. As used herein, a “mobilewireless device” refers to a device that is portable and communicateswirelessly. In some embodiments, a user wears or carries the mobilewireless device on the user's person or in the user's vehicle. Examplesof mobile wireless devices include mobile or cellular phones, wearabledevices (e.g., smart watch, fitness tracker, wearable sensor, smartglasses, etc.).

As used herein, a “pertinent sensor” refers to an electronic deviceincluding at least one sensing component (e.g., a sensor) and identifiedas having or potentially having information pertaining to an emergencybased on attributes of the electronic device. An electronic device maybe identified as a pertinent sensor based on attributes of theelectronic device including, but not limited to: device type, location,manufacturer, associated user(s), sensor(s), battery level, and powerstatus (e.g., powered on or powered off; active or inactive). A “set ofpertinent sensors” refers to one or more electronic devices (e.g.,sensors) identified as pertinent sensors having or potentially havinginformation pertaining to the same emergency. In some embodiments, theelectronic device is identified as a pertinent sensor with respect to anemergency based on an analysis and/or comparison of at least oneattribute of the device with information for the emergency. In someembodiments, an electronic device is identified as a pertinent sensorbased on a determination that the location of the device is in proximityto the location of the emergency. For example, a device is determined tobe in proximity to the location of the emergency when their respectivelocations are within a threshold distance from each other. In someembodiments, the threshold distance is no more than about 10 meters, 20meters, 30 meters, 40 meters, 50 meters, 60 meters, 70 meters, 80meters, 90 meters, 100 meters, 150 meters, 200 meters, 250 meters, 300meters, 350 meters, 400 meters, 450 meters, 500 meters, 600 meters, 700meters, 800 meters, 900 meters, or no more than about 1000 meters.

As used herein, a “relevant sensor” refers to an electronic devicehaving at least one sensing component (e.g., a sensor) determined tohave useful (or relatively useful) information pertaining to anemergency based at least in part on information, data, or multimediagathered by the sensing component(s) of the electronic device. Anelectronic device may be determined to be a relevant sensor based oninformation, data or multimedia including, but not limited to: audio,video, pictures, temperature data, health data, and environmental data.A “set of relevant sensors” refers to one or more electronic devices(e.g., sensors) determined to have useful (or relatively useful)information pertaining to the same emergency. In some embodiments, a setof relevant sensors is selected from a set of pertinent sensors (e.g., asubset of pertinent sensors).

As used herein, an “associated device” refers to a communication devicethat is associated with an electronic device. For example, a user may beusing several communication devices such as a mobile phone, a wearable,a home security system, a car computer. The user may have registeredthese devices with his or her account(s) and linked these devices with auser name, user number(s), email address(es), home or other physicaladdress(es). In some embodiments, associated devices may includecommunication devices of at least one additional user who is associatedwith user, e.g., a husband and wife, a father and son, a patient anddoctor, friends, work colleagues, etc. In some cases, the user may haveadded the second user as an emergency contact, a primary contact, asecondary contact, or a member of a group (e.g., part of the same club,organization, or workplace). In some cases, user may have agreed toshare location and other data with the second user. In some embodiments,the second user may be someone who is frequently contacted by the userand the communication device identifies the second user from the“Recently called” or “Frequently called” list. In some embodiments, theassociated devices may be devices that are proximal or near-by to thetriggering device such as obtained through a Wi-Fi scan. In someembodiments, an associated device is proximal to the triggering devicewhen the location of the associated device is within 1, 5, 10, 15, 20,25, 30, 35, 40, 45, 50, 60, 70, 80, 90, 100, 200, 300, 400, or 500meters of the location of the triggering device.

As used herein, the “list of associated devices” refers to a list ofcommunication devices that are associated with the user or thetriggering device (e.g., a second resident in a smart home). The list ofassociated devices may be listed by user name, phone number, emailaddress, physical address, coordinates etc. The device entry in the listmay include phone number, email address, physical address, coordinates,BSSID, SSID or MAC address. The list may be user defined or generated bythe device or the EMS. An entry in the list of associated devices mayalso be referred to as an account associated with the user.

As used herein, an “emergency alert” refers to a communication relatingto an emergency or non-emergency situation. In some embodiments, anemergency alert is an emergency request for assistance (e.g., therequest is associated with an emergency situation). In some embodiments,an emergency alert comprises an emergency indication. In furtherembodiments, an emergency indication is selected from one or more of thegroup consisting of traffic accident, police emergency, medicalemergency, and fire emergency. In some embodiments, an emergency alertis associated with a non-emergency situation (e.g., request for a towtruck after car breaks down). In some embodiments, an emergency alert isassociated with a device sending the alert. In other embodiments, anemergency alert is associated with a device not sending the alert (e.g.,a proxy request on behalf of a second device and/or a member device in agroup of devices). As used herein, an emergency alert is “associated”with a device or user when the emergency alert relates to an emergencyor non-emergency situation involving the device or user. In someembodiments, an emergency alert comprises data associated with a device(or user thereof). In some embodiments, an emergency alert comprisesdata associated with an electronic device sending the alert or anotherdevice. For example, in some embodiments, an emergency alert comprisesdata associated with a device, wherein the data set comprises currentand/or past location data. In another example, the data set comprisescurrent and/or past health data associated with the user of anelectronic device. In other embodiments, an emergency alert is sentand/or received separately from data associated with a device. Forexample, in some embodiments, an alert is sent first, and the recipientsubsequently queries the device that sent the alert for data associatedwith the emergency and/or device or user involved in the emergency aspart of an emergency flow script.

As used herein, a “first responder” refers to any person or personsresponsible for addressing an emergency situation. A first responder mayalso be referred to as an “emergency responder.” In some embodiments, afirst responder refers to government personnel responsible foraddressing an emergency situation. In some embodiments, a firstresponder is responsible for a particular jurisdiction (e.g., amunicipality, a township, a county, etc.). In some embodiments, a firstresponder is assigned to an emergency by an emergency dispatch center(hereinafter, “EDC”). In some embodiments, a first responder responds toa request for emergency assistance placed by a user via a usercommunication device. In some embodiments, a first responder includesone or more firefighters, police officers, emergency medical personnel,community volunteers, private security, security personnel at auniversity, or other persons employed to protect and serve the publicand/or certain subsets of the population.

As used herein, an “emergency service provider” (ESP) is a public orprivate organization or institution responsible for providing emergencyservices. For example, in some embodiments, an EDC (e.g., a PSAP, asdescribed above), a fire department, a police department, and a hospitalmay all be considered emergency service providers. In some embodiments,an emergency responder is a member of an ESP.

As used herein, a “recipient” refers to one or more persons, services,or systems that receive a request for assistance (e.g., an emergencyalert). The recipient varies depending on the type of request. In someembodiments, a recipient is an emergency service. In some embodiments, arecipient is an emergency service when the request for assistancepertains to an emergency (e.g., a tier 2 emergency). In someembodiments, a recipient is an emergency management system. In someembodiments, a recipient is an emergency dispatch center (e.g., a publicsafety answering point or PSAP). In some embodiments, a recipient is anemergency dispatch center, wherein the request is first routed throughan emergency management system (e.g., request is sent to the EMS, butultimately is sent to an EDC). In some embodiments, a recipient is afirst responder (e.g., a communication device of a first responder). Insome embodiments, a recipient is an associated device of a user or anaccount associated with the user. In some embodiments, a recipient is anon-emergency service or personnel, for example, a relative or friend.In such situations, a user of a communication device (or member deviceor second device) may not require emergency assistance, but does needhelp.

As used herein, a “user” refers to one or more person or personsassociated with a system, server, or device (e.g., electronic device,member device, second device, device of a first responder, etc.). Insome embodiments, a user is an administrator and/or authorized user whohas authorization for generating or customizing an emergency flowscript. In some embodiments, the administrator and/or authorized userworks for or acts on behalf of an organization that utilizes thesystems, servers, devices, methods, and media of the instant applicationfor managing emergency communications. In some embodiments, theorganization is a public or private organization. In some embodiments,the organization provides a transportation service (e.g., taxi company,ride-sharing company, shipping company, railroad company, etc.). In someembodiments, a user utilizes a device to send an emergency alert orrequest for assistance. In some embodiments, user refers to one or morepersons who are paid subscribers of a network access service, forexample, cellular service subscribers. In some embodiments, a userrefers to anyone who gains access to a network via a router, forexample, a Wi-Fi router, and is not a paid subscriber of any accessservice. In some embodiments, a device associated with a user is adevice carried or worn on the person of the user (e.g., a phone orwearable device). In some embodiments, a device associated with a useris not carried or worn on the person of the user (e.g., a home securitysensor or camera installed in the home of the user, a vehicle trackingsystem installed in a vehicle of the user, etc.).

As used herein, “data” refers to a collection of information about oneor more entities (e.g., user of a user communication device) and/or anenvironment that pertains to characteristics of the one or moreentities. In some embodiments, an entity is a person such as a user. Insome embodiments, an entity is a thing (e.g., a house). For example, insome embodiments, data comprises sensor data from home sensorsassociated with a house. In this example, the data is also associatedwith one or more persons (e.g., the homeowner(s) and/or inhabitant(s)).In some embodiments, data refers to meta-data. In some embodiments, datacomprises health information about the user of a communication device.In some embodiments, data comprises information about the surroundingenvironment of the user of the user communication device (e.g.,surrounding temperature, location, elevation, barometric pressure,ambient noise level, ambient light level, surrounding geography, etc.).In some embodiments, data comprises information about other users thatis pre-stored in a device or in a database (e.g., a database within agroup of devices who are related to the user of the user communicationdevice as predefined by the user). In some embodiments, the data setcomprises information from two or more users of user communicationdevices, wherein each user is affected by an emergency situation. As anexample, two unrelated users are involved in a vehicular collision, andeach user sends a separate emergency alert (for traffic accident) usinghis/her communication device. In this example, the separate emergencyalerts are associated (e.g., by an emergency management system and/oremergency dispatch center) with the same emergency based on theproximity of time, location, and emergency indication of the emergencyrequests. As a result, the data set for this accident comprisesinformation from both user communication devices. In this example, thedata set comprises location data from both devices (e.g., GPScoordinates), biosensor data for one or both devices (e.g., biosensordata such as heart rate and blood pressure can be important in case ofinjury), and information about the vehicle driven by each user (e.g.,make, model, and year of manufacture information stored on the device).In some embodiments, data comprises current data. In furtherembodiments, current data comprises information that is equal to or lessthan 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 35, 40, 45, 50, 55, or 60minutes old, including increments therein. In further embodiments,current data comprises information that equal to or less than 1, 2, 3,4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22,23, or 24 hours old. In some embodiments, data comprises historicaldata. In further embodiments, historical data comprises information thatis equal to or more than 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 35, 40,45, 50, 55, or 60 minutes old, including increments therein. In furtherembodiments, historical data comprises information that equal to or morethan 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,20, 21, 22, 23, or 24 hours old. In some embodiments, the age ofinformation is calculated from the date the information is firstcollected (e.g., when a sensor first detects a sensed parameter such as,for example, heart rate).

As used herein, “health data” refers to medical information associatedwith a user of a device. In some embodiments, health data comprisesmedical history such as, for example, past illnesses, surgery, foodand/or drug allergies, diseases, disorders, medical diagnosticinformation (e.g., genetic profile screen), or any combination thereof.In some embodiments, health data comprises family medical history (e.g.,family history of breast cancer). In some embodiments, health datacomprises current health information such as, for example, currentsymptoms, current medications, and/or current illnesses or diseases. Insome embodiments, health data comprises user age, height, weight, bloodtype, and/or other biometrics. In some embodiments, medical historycomprises medical information that is equal to or more than 1, 2, 3, 4,5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23,or 24 hours old. In some embodiments, medical history comprises medicalinformation that is equal to or more than 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,29, or 30 days old. In some embodiments, current health informationcomprises information that is equal to or less than 1, 2, 3, 4, 5, 6, 7,8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, or 24hours old. In some embodiments, current health information comprisesmedical information that is equal to or less than 1, 2, 3, 4, 5, 6, 7,8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,26, 27, 28, 29, or 30 days old.

As used herein, “user data” refers to general information associatedwith a user of a device. In some embodiments, user data comprises useridentity, user name, height, weight, eye color, hair color, ethnicity,national origin, religion, language(s) spoken, vision (e.g., whetheruser needs corrective lenses), home address, work address, occupation,family information, user contact information, emergency contactinformation, social security number, alien registration number, driver'slicense number, vehicle VIN, organ donor (e.g., whether user is an organdonor), or any combination thereof. In some embodiments, user data isobtained via user input.

As used herein, “sensor data” refers to information obtained or providedby one or more sensors. In some instances, a sensor is associated with adevice (e.g., user has a communication device with a data link viaBluetooth with a wearable sensor, such as, for example, a heart ratemonitor or a pedometer). Accordingly, in some embodiments, the deviceobtains sensor data from the sensor (e.g., heart rate from the heartrate monitor or distance traveled from the pedometer). In someinstances, the sensor data is relevant to an emergency situation (e.g.,heart rate during a cardiac emergency event). In some embodiments, asensor and/or sensor device comprises an acoustic sensor, abreathalyzer, a carbon dioxide sensor, a carbon monoxide sensor, aninfrared sensor, an oxygen sensor, an ozone monitor, a pH sensor, asmoke detector, a current sensor (e.g., detects electric current in awire), a magnetometer, a metal detector, a radio direction finder, avoltage detector, an air flow meter, an anemometer, a flow sensor, a gasmeter, a water meter, a Geiger counter, an altimeter, an air speedindicator, a depth gauge, a gyroscope, a compass, an odometer, a shockdetector (e.g., on a football helmet to measure impact), a barometer, apressure gauge, a thermometer, a proximity sensor, a motion detector(e.g., in a home security system), an occupancy sensor, or anycombination thereof, and in some embodiments, sensor data comprisesinformation obtained from any of the preceding sensors. In someembodiments, one or more sensors are physically separate from a userdevice. In further embodiments, the one or more sensors authorize theuser device to obtain sensor data. In further embodiments, the one ormore sensors provide or send sensor data to the user deviceautonomously. In some embodiments, the user device and the one or moresensors belong to the same group of devices, wherein member devices areauthorized to share data. In some embodiments, a user device comprisesone or more sensors (e.g., user device is a wearable device having asensor or sensing component).

As used herein, “communication link” refers to a communication pathwayfrom a device (e.g., communication device) to another device or to anintermediate device (e.g., a router) such as over a network. In someembodiments, the communication device establishes a communication linkwith another device or an intermediate device to transfer information(e.g., a location of the device) or to obtain information from arecipient such as, for example, location of a first responder assignedto a request for assistance associated with the communication device(e.g., device of first responder). In some embodiments, a communicationlink refers to the point-to-point communication channels, point-to-pointand end-to-end data sessions, and/or the physical hardware facilitatingthe communication channel(s) (e.g., antennas used tocommunicate/transmit information). In some embodiments, a data sessioncomprises session parameters and the network route taken from one deviceto another device.

As used herein, a “data channel” refers to a communication sessionbetween two devices wherein data packets are exchanged between thedevices. In some embodiments, a data session is setup using exchange ofcertain data packets, also called as “handshake signals,” which are ableto define the capabilities of the data session. For example, in someembodiments, the data session “handshake” provides for the ability totransfer multi-media data, voice data, and other data via the datasession. In some embodiments, the data session is setup without the useof handshake signals, wherein the two devices involved share datapackets according to a predefined protocol (e.g., a previously agreedupon protocol). In some embodiments, the data session is routed throughan EMS, which stores the multi-media, voice, and/or other data from anyof the devices that are part of the data session. In furtherembodiments, the EMS shares the data from the data session with theother device (e.g., device of a first responder). In some embodiments,the EMS manages the data session.

Modern communication devices, for example, smart phones, tabletcomputers, wearable communication devices, smart sensor devices and/orsystems are often equipped with a variety of features for determininglocation information of the communication device using, for example,GPS, or triangulation with cellular phone towers. Modern communicationdevices also often include functionality to store data regarding a userof the communication device, for example, health information about theuser.

In some embodiments, the communication device (or communication moduleof the device) communicates with a recipient through one or more datachannels. In some embodiments, the recipient is an emergency managementsystem. In some embodiments, the EMS routes communications to an EDC. Infurther embodiments, the EMS establishes a first data channel with thecommunication device and a second data channel between the EMS and theEDC, wherein the EMS bridges the first and second data channels toenable the communication device and the EDC to communicate. In someembodiments, the EMS converts data (e.g., data set) from thecommunication device into a format suitable for the EDC (e.g., analog ordigital, audio, SMS, data, etc.) before sending or routing the formatteddata to the EDC. In some embodiments, the EMS routes communications to adevice associated with a first responder. In some embodiments, thecommunication device relays additional communications, information,and/or data sent or shared between member devices in the group ofdevices to the EMS or EDC after a request for assistance has been sent.In further embodiments, the additional information is relayed to the EMSor EDC after the request for assistance has been sent in order toprovide current information that is relevant to the request. Forexample, in some instances, communications between member devicescontain information relevant to the emergency (e.g., information thatthe user of member device who is experiencing a medical emergencysuffers from diabetes). Accordingly, in some embodiments, theinformation is sent autonomously, at request of a user of thecommunication device, or at request of the recipient (e.g., EMS, EDC,first responder, etc.).

The following illustrative examples are representative of embodiments ofthe invention described herein and are not meant to be limiting in anyway.

EXAMPLES

Just In Time, an emergency response company, aids emergency serviceproviders (such as public safety answering points, or “PSAPs”) bygathering emergency data from a variety of sources and delivering thedata directly to the emergency service providers. Traditionally, PSAPsare only technologically capable of receiving telephone calls (e.g.,9-1-1 emergency calls) with no additional data. Thus, when an emergencycall is made to a PSAP from a mobile phone (with a dynamic and uncertainlocation), PSAP operators or call-takers must speak directly to thecaller to determine the caller's location and the nature of the caller'semergency. Unfortunately, many people involved in emergency situationsare unable to articulate their location or may not even know—and even ifthey do, the time spent articulating their location to the PSAP operatorcan often be the difference between life and death. Similarly, PSAPoperators are forced to respond to emergencies with little or noinformation about the persons involved (e.g., health data or medicalhistories) or context of the emergencies (e.g., type of emergency,audio/video of the surroundings, etc.). Just In Time knows just howcritical it is to quickly and accurately provide locations andsituational/contextual information during emergencies to emergencyservice providers.

To aid emergency service providers (hereinafter, “ESPs”), Just In Timemaintains and provides an emergency management system (hereinafter,“EMS”) that receives and stores data, information, and multimedia from aplurality of sources, such as mobile phones and mobile applications,internet of things (IoT) devices, intelligent vehicle systems, and otherelectronic devices. During an emergency the EMS can gather data,information, and multimedia from multiple sources in real-time andintelligently determine which of the sources (and, by extension, whichof the data, information, and multimedia) are the most relevant to theemergency. The EMS can then provide ESPs with the most relevantemergency data without bogging down response times, as ESPs are oftenunder tremendous pressure, and the decisions they make must be quick.Therefore, great care must be taken to avoid overwhelming an ESP withthe wrong information or too much information.

Henry, a middle-aged man, and his wife Michelle live together in a housein Boston. Henry has bought and outfitted their house with two Nest Camdevices (smart cameras), one in the kitchen and one in the living room,and an Amazon Echo (smart speaker) located in the living room. Michelleowns an Apple Watch (smart watch) that she wears every day. One day,Henry collapses in the living room while on his way to the kitchen for aglass of water. From the bedroom, Michelle hears Henry fall and runsover to check on him. Distraught, Michelle yells “Alexa, call 9-1-1!”The Amazon Echo recognizes Michelle's voice command, and, in response toreceiving the voice command, generates an emergency alert and initiatesa 9-1-1 call to a public safety answering point (PSAP). The emergencyalert, which includes the location of the Amazon Echo and the emailaddress Henry used to set up the Amazon Echo, is instantly received byJust In Time's EMS. When the PSAP receives the 9-1-1 call, the PSAPsends an identifier of the 9-1-1 call (e.g., the inbound phone number)to the EMS, which the EMS uses to associate the PSAP with the emergencyalert.

In response to receiving the emergency alert, the EMS attempts to detectand identify all sensor devices pertinent to Henry's fall. First, theEMS queries Nest's network system with Henry's email address (receivedin the emergency alert) for devices associated with Henry's emailaddress. Nest's network system returns identifiers and informationregarding both the Nest Cam in Henry and Michelle's kitchen and the NestCam in their living room. The EMS recognizes the locations of the NestCams to be in the vicinity of the location of the Amazon Echo.Additionally, the EMS prompts the Amazon Echo to broadcast a distresssignal to all sensor devices in the vicinity. Michelle's Apple Watchreceives the distress signal from the Amazon Echo and returnsinformation regarding the Apple Watch and a signal to the Amazon Echoindicating that the Apple Watch is active and available, which are thenrelayed to the EMS by the Amazon Echo. The EMS thus identifies the foursensor devices—the Amazon Echo, both Nest Cams, and Michelle's AppleWatch—as a first set of sensors pertinent to the emergency (e.g.,Henry's fall).

Then, after identifying the Amazon Echo, Nest Cams, and Apple Watch assensors pertinent to the emergency, the EMS connects to each of the fourdevices using the information received regarding the devices. In thiscase, the EMS establishes a direct communication link with the AmazonEcho and the Apple Watch and establishes an indirect communication linkwith the two Nest Cams, wherein the EMS can query the Nest network toreceive data from the Nest Cams. The EMS then receives multimediacontents including an audio stream from the Amazon Echo, a second audiostream from the Apple Watch, a video stream from the Nest Cam in thekitchen, and a second video stream from the Nest Cam in the living room.As the 9-1-1 call initiated by the Amazon Echo commences, the EMSanalyzes the multimedia received from each of the sensor devices in realtime to determine which of the sensors is the most relevant to theemergency.

A PSAP call-taker answers Michelle's 9-1-1 call through a computer aideddispatch (CAD) system at the PSAP and asks (through the Amazon Echospeakers) for the nature of the emergency. Michelle responds by sayingaloud that her husband has fallen and can't get up. Concurrently, Henrybegins moaning “my back, my back, I can't move my back.” The Amazon Echoreceives Michelle's response to the PSAP call-taker, but Henry's moansare too faint to be picked up by the Amazon Echo's microphones. However,Michelle's Apple Watch, positioned close to both Michelle and Henry, isable to pick up Henry's moans as well as Michelle's responses to thePSAP call-taker. Using voice and speech recognition software to analyzethe audio streams received from the Amazon Echo and the Apple Watch, theEMS recognizes two different voices (Michelle's and Henry's) in theaudio stream received from the Apple Watch, and only one voice in theaudio stream received from the Amazon Echo. The EMS also identifies morekey words and key terms in the audio stream received from the AppleWatch than in the audio stream received from the Amazon Echo. The EMSthus concludes that the audio stream received from the Apple Watch ismore relevant to the emergency than the audio stream received from theAmazon Echo. Concurrently, the EMS uses video analysis software toanalyze the video stream received from the kitchen Nest Cam and theliving room Nest Cam. The EMS identifies two humans in the video streamreceived from the living room Nest Cam and none in the video streamreceived the kitchen Nest Cam. The EMS thus concludes that the videostream received from the living room Nest Cam is more relevant to theemergency than the video stream received from the kitchen Nest Cam.

Furthermore, through the analysis of the audio streams (e.g., naturallanguage processing of an audio transcription), the EMS determines thatthe nature of the emergency is likely a medical emergency. The EMS thenapplies a prioritization rule wherein audio streams are prioritized overvideo streams during a medical emergency, and ultimately concludes thatthe Apple Watch is the sensor most relevant to the emergency. Inresponse to identifying that there are sensors pertinent to theemergency, the EMS sends an indicator of available multimedia to thePSAP. The PSAP call-taker then sends a request for multimedia to theEMS, and the EMS transmits the audio stream received from the AppleWatch to the PSAP, which the PSAP call-taker uses to determine thatHenry has suffered a back injury. The PSAP call-taker uses the CADsystem to dispatch an ambulance to Henry and Michelle's location and,because of the multimedia received from the EMS, is able to tell thefirst responders in the ambulance that they will be dealing with a backinjury. The PSAP call-taker additionally opts to transmit the audiostream received from the Apple Watch to the first responders in theambulance, so that the first responders may have additional situationalawareness regarding the emergency as they navigate to Henry andMichelle's home.

While preferred embodiments of the present invention have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. Numerousvariations, changes, and substitutions will now occur to those skilledin the art without departing from the invention. It should be understoodthat various alternatives to the embodiments of the invention describedherein may be employed in practicing the invention. It is intended thatthe following claims define the scope of the invention and that methodsand structures within the scope of these claims and their equivalents becovered thereby.

What is claimed is:
 1. A computer-implemented system for transmitting multimedia from an electronic device to an emergency service provider (ESP), the system comprising: a) a network server comprising one or more processors; and b) an emergency management system (EMS) executed on the network server and configured to: i) identify an electronic device as being associated with an emergency alert; ii) deliver a multimedia inquiry to the electronic device; iii) receive a confirmation of the multimedia inquiry from the electronic device; iv) determine the emergency service provider (ESP) to receive multimedia from the electronic device based on a location of the electronic device and a geofence system; v) establish a communication link between the electronic device and the ESP; and vi) transmit multimedia content from the electronic device to the ESP via the communication link.
 2. The system of claim 1, wherein the multimedia content comprises a video feed.
 3. The system of claim 2, wherein the EMS is configured to transmit the multimedia content from the electronic device to the ESP via the communication link by: a) hosting the video feed at a remote server; and b) providing the ESP with a web link to the video feed hosted on the remote server.
 4. The system of claim 1, wherein the EMS is configured to identify the electronic device as associated with the emergency alert by autonomously receiving the emergency alert from the electronic device when the electronic device executes an emergency phone call.
 5. The system of claim 1, wherein the EMS is further configured to: a) display a multimedia access button within an emergency response application executed on a computing device at the ESP; b) establish the communication link between the electronic device and the ESP in response to receiving selection of the multimedia access button; and c) display the multimedia content through the emergency response application.
 6. The system of claim 1, wherein the EMS is further configured to: a) display a multimedia access button within an emergency response application executed on a computing device at the ESP; b) receive selection of the multimedia access button; and c) deliver the multimedia inquiry to the electronic device in response to receiving the selection of the multimedia access button.
 7. The system of claim 6, wherein the emergency response application is a web application accessible via a web browser using a URL.
 8. The system of claim 6, wherein the EMS is configured to identify the electronic device as being associated with the emergency alert by receiving an emergency data request from the ESP, wherein the emergency data request is generated and transmitted by the emergency response application in response to receiving selection of the multimedia access button and wherein the emergency data request comprises an identifier of the electronic device.
 9. The system of claim 1, wherein the emergency alert is an emergency phone call made from the electronic device and wherein the EMS is configured to identify the electronic device as associated with the emergency alert by autonomously detecting the emergency phone call made by the electronic device.
 10. The system of claim 1, wherein the EMS is configured to determine the ESP to receive multimedia from the electronic device by: a) retrieving a plurality of geofences associated with a plurality of ESPs including the ESP; and b) determining that the location of the electronic device is within a geofence associated with the ESP.
 11. The system of claim 10, wherein the EMS is further configured to: a) receive credentials associated with an account of an ESP user through an emergency response application executed on a computing device at the ESP; b) identify an ESP ID associated with the account of the ESP user; and c) determine that the geofence associated with the ESP is associated with the ESP ID.
 12. The system of claim 1, wherein the multimedia inquiry is an SMS message comprising a web link and wherein confirmation of the multimedia inquiry comprises selection of the web link.
 13. The system of claim 1, wherein the multimedia inquiry is a push notification and wherein confirmation of the multimedia inquiry comprises selection of the push notification.
 14. The system of claim 1, wherein the EMS is further configured to: a) sample a connection quality of the transmission of the multimedia content; and b) in response to the connection quality falling below a threshold value, downgrading the multimedia content.
 15. The system of claim 14: a) wherein the multimedia content comprises a video feed; and b) wherein the video feed is downgraded to image stills or reduced framerate or resolution.
 16. The system of claim 14: a) wherein the multimedia content comprises a video file; and b) wherein the video file is downgraded to a summarization.
 17. The system of claim 14: a) wherein the multimedia content comprises an audio feed or audio file; and b) wherein the audio feed or audio file is downgraded to a text transcription.
 18. The system of claim 1, wherein the EMS is further configured to determine an access level assigned to the electronic device and transmit the multimedia inquiry to the electronic device in response to determining that the EMS does not have permission to access the electronic device based on the access level assigned to the electronic device.
 19. A method for transmitting multimedia from an electronic device to an emergency service provider (ESP) by an emergency management system (EMS), the method comprising: a) identifying an electronic device associated with an emergency alert; b) delivering a multimedia inquiry to the electronic device; c) receiving a confirmation of the multimedia inquiry from the electronic device; d) determining an emergency service provider (ESP) to receive multimedia from the electronic device using a location of the electronic device and a geofence system; e) establishing a communication link between the electronic device and the ESP; and f) transmitting multimedia content from the electronic device to the ESP via the communication link.
 20. A method for transmitting multimedia from an electronic device to an emergency service provider (ESP) by an emergency management system (EMS), the method comprising: a) detecting an emergency call made from an electronic device; b) in response to detecting the emergency call made from the electronic device, delivering a multimedia inquiry to the electronic device; c) receiving a confirmation of the multimedia inquiry from the electronic device; d) obtaining a location of the electronic device; e) determining an emergency service provider (ESP) to receive multimedia from the electronic device based on the location of the electronic device; f) displaying a multimedia access button within an emergency response application executed on a computing device at the ESP; and g) in response to receiving selection of the multimedia access button, establishing a communication link between the electronic device and the ESP and transmitting multimedia content from the electronic device to the ESP via the communication link. 